Skip to content
BenchCove
Dashboard

How to remove duplicate rows from a CSV file, by email or any column

Updated

Duplicates usually come from merging exports, re-importing a list or a form that was submitted twice. The tricky part isn't deleting them; it's deciding what counts as a duplicate and which copy to keep.

First decide what a duplicate is

Two rows can be identical in every column, or only share a key such as an email address or customer ID. If you match on every column, two rows for the same person with different phone numbers both stay. If you match on email, only one of them stays, and you lose the other row's phone number.

Also check for near-duplicates: “Ana@Mail.com” and “ana@mail.com ” (with a trailing space) look the same to a person but are different text to most tools.

In Excel

  1. Open the file and select the data (or click inside it).
  2. Go to Data → Remove Duplicates.
  3. Tick only the columns that define a duplicate, for example Email, and click OK.
  4. Excel keeps the first occurrence and deletes the rest. Spaces are not ignored, so trim them first (for example with the TRIM function in a helper column).

Opening a CSV directly in Excel can change values: leading zeros in IDs and phone numbers disappear and some text turns into dates. Import it with Data → From Text/CSV if that matters.

In Google Sheets

  1. Select the data range.
  2. Go to Data → Data cleanup → Trim whitespace first, if spaces might differ.
  3. Then Data → Data cleanup → Remove duplicates, tick “Data has header row”, choose the columns to compare, and confirm.

The private way: in BenchCove

Free, no sign-up. Uses the File & Data Workbench. Nothing is uploaded.

  1. Open the File & Data Workbench and import your CSV, TSV or Excel file. Nothing is uploaded; it is read in your browser.
  2. Go to Clean and choose Remove duplicates.
  3. Compare on all columns or pick key columns such as Email. Optionally ignore spaces at the start and end and ignore upper and lower case, and choose whether to keep the first or last row.
  4. Check the preview: every row that will be removed is marked, with the row it duplicates. Apply it, and undo at any time.
  5. Export a clean CSV. Your original file is never changed.
Remove duplicates in your browser

Questions

Which copy is kept when duplicates are removed?

Excel and Google Sheets keep the first occurrence. In BenchCove you choose the first or the last row in the file.

Can I remove duplicates based on one column only?

Yes. Choose only that column (for example Email) as the key. Rows that share it are treated as duplicates even if other columns differ, so check what you'd lose first.

Is it safe to do this with customer data?

Doing it in a desktop spreadsheet or in BenchCove keeps the file on your computer. Avoid online tools that upload the file to their server if the data is personal.