Remove Duplicate Images From Your Computer Using Lightroom
March 4, 2014
5 comment

Has this ever happened to you? You download images from a website multiple times because you forgot where you download it to. You export images multiple times from Lightroom because you forgot where you exported it to. What you wind up with is duplicate images scattered across your hard drive and no easy way to clean them up.

Until now!

In this tutorial, I will show you how to use Lightroom to clean up those duplicate images from your hard drive. Just in time for spring cleaning!

Disclaimer:  This tutorial isn’t really meant for your original processed images since the catalog contains all the edit data. Even if you use XMP sidecar files, you might lose important information. This tutorial is more for removing duplicate exported, downloaded, or stock images. Remember to always back up your catalog before doing bulk moves!

Ok, the disclaimer is out of the way, so let’s get started!

Setup

For this tutorial, I created three folders of stock images. Each folder had some unique images as well as images that were duplicated in other folders.

Lightroom Fanatic - Duplicate Images On Hard Drive

Lightroom Fanatic – Duplicate Images On Hard Drive

I also created an empty catalog to highlight the process and because I didn’t want to clutter up my working catalog.

Empty Lightroom Catalog

Lightroom Fanatic – Empty Lightroom Catalog

Importing

The magic of Lightroom comes in to play during the import process. Basically, we’re going to leverage Lightroom’s duplicate recognition to import only one copy of an image. Lightroom will recognize duplicates by comparing all the images being imported and also by comparing the incoming images against what is already in the catalog. In this tutorial, since I’m starting with an empty catalog, I’m relying only on Lightroom comparing all the incoming images for duplicates.

In the import dialog below, I’ve navigated to the top level folder that contains the three stock image folders I created in the setup. As you can see, there are some duplicate images in the list, including the iTunes logos and the iLife logo.

Import With Duplicates

Lightroom Fanatic – Import With Duplicates

When I click on the Don’t Import Suspected Duplicates checkbox, Lightroom will gray out any images that it suspects are duplicates.

Don't Import Suspected Duplicates

Lightroom Fanatic – Don’t Import Suspected Duplicates

Now, Lightroom has grayed out the copies of the iLife and iTunes logos, along with the other images that have multiple copies.

When I click on import, Lightroom will only import the images that are not grayed out, which means I’m getting only one copy of an image.

Imported unique images

Lightroom Fanatic – Imported Unique Images

Technically, the duplicate images still exist on my hard drive, but Lightroom has only imported pointers to one copy of each image. If you notice the folder list on the left, it has a handful of images from each folder.

Moving The Unique Images To New Folder

Now I’m going to move the physical image files in to a new folder so that I have one folder with a unique set of images. To do this, I select all the images in the Lightroom catalog and then right-click on the top-level folder. I created a folder named depulicated (somehow I butchered the deduplicated title) and checked the Include selected photos option.

Create folder with selected photos

Lightroom Fanatic – Create Folder With Selected Photos

Lightroom will create the new folder and move the unique image files in to the new folder, which will also be reflected in your catalog.

New deduplicated folder

Lightroom Fanatic – New De-Duplicated Folder

Checking my hard drive, I can see the depulicated folder, which contains the unique image files.

Lightroom Fanatic - Unique Images

Lightroom Fanatic – Unique Images In New Folder

Deleting The Duplicates

What’s left over in the stock image folders are all the duplicate images, which I can now safely delete.

Duplicate image fallout

Lightroom Fanatic – Duplicate Images Fall Out

That’s all there is to it!




 

, , , , ,

How To Automatically Launch Lightroom By Inserting A Media Card

FacebookTwitterGoogle+PinterestLinkedIn Has this e...

Read more

Multiple Computer Lightroom Workflow Using Catalog Export

FacebookTwitterGoogle+PinterestLinkedIn Has this e...

Read more

How To Find Your TIFF (or PSD) Files In Your Lightroom Catalog

FacebookTwitterGoogle+PinterestLinkedIn Has this e...

Read more

There are 5 comments

  • raden adams says:

    Hey David, This is quite funny and amazing really as I have just read on of your tutorials, left you a fairly long comment and at one point asked if you knew of a good duplicate finder plug-in for LR as I have managed to have hundreds scattered about my catalog. Well, I wen’t to your Twitter site and immediately read this tweet about this duplicate finder!! So, thank you and I will give it a try!
    Raden Adams

    • dave says:

      Thanks again, Raden, for the comment! I saw your other comment as well about duplicates already in the catalog. Stay tuned on that one, I’ll have a post up soon!

  • raden adams says:

    How do I find and remove duplicates that are already scattered about in my LR5 catalog? Also, I am just now starting to use LR correctly and for the last 2 or 3 years, I was very inconsistent in using keywords and other metadata that would allow me to find my images quickly. My catalog, over 45,000 images is an absolute mess and would take forever to correctly organize so I am doing a little at a time. What about if I created a new LR5 catalog, I started a better backup system and I am now transferring my current catalog on 2 hard drives and a cloud service for better protection from loss. Once I have all files transferred, I want to remove all the files from my computers internal hard drive because it’s full now and I am having lots of problems with it’s performance because it is packed full. Ideally, I would like to create a new catalog and then I can find and deal with the duplicates as I import new files. Also, I want to continue to use the 2 hard drive system and not store any images at all on my computer to optimize performance and speed from now on. When I import new images in the future, I will use one hard drive as my catalog working drive and select the option for LR to send a second copy to the other hard drive on import. They will all be raws without edits but I will back up my working catalog regularly to keep it up to date as I have just recently learned about the catalogs, where they all are stored, ircat is main catalog and discovered the back up catalogs and that some of them are in different locations. What do you think about any and all of this or would you recommend some entirely different way of doing this? Thank you very much. Raden

    • dave says:

      Hi again, Raden. I’ve got a post in the works to answer the duplicate image question. As for the other piece, I was in the same place with a full hard drive. What worked for me was importing in to LR, doing all my edits, exporting my final images, and then when I was done with a project, I would migrate the original raw files, the edits, and the final images to my external hard drive.

      One way to do this is to simply have a catalog that has both your local hard drive and external hard drive in it, and when you are done editing images on your local drive, drag the originals to the external drive in Lightroom. That should move the files and keep the edits stored in the catalog associated with them. That way, you still have access to those files.

      Another option is to use XMP sidecar files. When your images are ready to archive, you can drag and drop the folder outside of LR to your archive drive. If you want, you can have another catalog associated with the external drive and import the folder there. Since you have the XMP sidecar files, you’ll get your edits back, but lose things like virtual copies.

      Another more involved option is to export your final image sets to a new catalog and associate that with your originals, but I think the first two options are easier to manage.

      Hope that helps!

  • ivanden says:

    Thank you for this.
    I have one more suggestion to users, check this “DuplicateFilesDeleter”
    recently i found in internet after searching for duplicate files remove.