Deduplicate
Find and remove duplicate rows from your spreadsheet. Choose which columns define a duplicate, and keep either the first or last occurrence.
By Operelio team
On this page7
What it does
Scans your file for rows that share the same values in one or more key columns, then removes the duplicates. You choose which columns to check and whether to keep the first or last occurrence of each duplicate group.
How to use it
Upload your file
Any supported format (.xlsx, .xls, or .csv).
Pick key columns
Select one or more columns that define a unique row. For example, "email" alone, or "first_name" and "last_name" together.
Choose which to keep
First occurrence (default) or last occurrence.
Set matching mode
Exact matching is on by default. Pro adds similarity matching with a threshold slider for catching typos.
Run
Click Run. The output contains only unique rows.
What the results show
After running, the results page shows the total input rows, unique rows kept, duplicates removed, and the number of duplicate groups found (a group is two or more rows that match on your key columns).
You also see the settings that were used: which columns Operelio matched on, which keep strategy was applied (first or last), whether case was ignored, and whether whitespace was trimmed. If you ran in similarity mode, the threshold percentage shows too.
On Pro, a separate file with just the removed rows is available for download alongside your cleaned output, so you can review what was dropped before importing.
Exact vs. similarity matching
Exact matching (available on all plans) compares cell values character by character. Two rows are duplicates only if the key columns match perfectly.
Similarity matching (Pro, Team, and Agency) uses a similarity score from 0 to 100. You set a threshold with a slider, and any two rows with a similarity score at or above that threshold are treated as duplicates. This catches typos and minor variations like "Jon Smith" vs. "John Smith". The score measures how many character edits separate the two values, so closer values score higher.
The default threshold is 80, which works well for matching contact names. Go higher (90+) if you want to be more conservative and only catch very close matches. Go lower (70) for more aggressive matching that catches bigger variations.
Matching options
| Option | Default | What it does |
|---|---|---|
| Case insensitive | On | Treats "WESTMARCH" and "westmarch" as the same value. |
| Trim whitespace | On | Strips leading and trailing spaces before comparing. |
| Full row match | On | Default scope. Compares every column. Switch to specific columns to match on a subset (for example, email only). |
Duplicates export (Pro)
On Pro, Team, and Agency, Operelio writes a second file containing just the rows that were removed. This lets you review what was dropped without re-running the job. If no duplicates were found, no extra file is created.
Frequently asked questions
What if I want to match on every column?
Pick "Entire row". That's the default. It compares all columns and treats two rows as duplicates only if every cell matches. Switch to "Specific columns" if you want to match on a subset, like email only.
What if my file has more rows than my plan allows?
Operelio processes the first N rows up to your plan limit and shows a banner telling you how many rows were skipped. Free is 2,000 rows, Pro is 50,000, Team is 75,000, Agency is 150,000. To process more, upgrade your plan or use the Split File tool to break the file into smaller chunks.
What happens to my original file?
Nothing. Operelio reads your upload and writes a new file. Your original is untouched. The cleaned output is a separate download.
Can I use this in a workflow with other tools?
Yes, on Pro. Workflow Builder lets you chain Deduplicate with other tools in one job. A common combination is Clean Headers, then Deduplicate, then CRM Formatter, all in a single run.
What plan do I need?
Free covers exact matching and works on files up to 2,000 rows. Pro ($39/month) adds similarity matching and a separate file with the removed rows, and handles up to 50,000 rows per job. Team ($99/month) goes up to 75,000 rows. Agency ($299/month) goes up to 150,000 rows.
Ready to get started?
Upload a file and run your first transformation. Free, no credit card required.