Remove duplicates
Remove duplicate lines, blank lines and extra whitespace.
How to use
Paste a list, one item per line, and press Run. The result keeps the first occurrence of each line in its original position and drops every later repeat, so the order you pasted survives — unlike sorting, which changes it.
Three options decide what counts as the same line. Trim spaces, on by default, removes spaces, tabs and other whitespace at the start and end of each line, including non-breaking spaces, so “apple” and “apple ” become one. Remove blank lines, also on by default, drops empty lines; a line with only spaces counts as empty when Trim spaces is on. With it off, blank lines are treated like any other line, so only the first one is kept. Ignore uppercase/lowercase, off by default, treats Apple, apple and APPLE as the same line and keeps whichever came first.
Lines that look identical can still both survive, because something invisible differs inside them: a non-breaking space between two words, a curly apostrophe (’) instead of a straight one ('), a letter from another alphabet that looks the same, such as the Cyrillic о and the Latin o, or an accented letter stored as one character in one line and as a letter plus a separate accent in the other. Trim spaces only touches the ends of a line, so these differences remain.
Windows and Unix line breaks are both accepted. The result is plain text, one line per item, ready for the Copy button. Everything runs in your browser; the list is not sent anywhere.
Example
apple / banana / apple / cherry / bananaapple, banana, cherry: the first of each, in the original order.“apple” and “apple ” (trailing space)One line with Trim spaces on, two with it off.Apple and appleTwo lines by default; one, Apple, with Ignore uppercase/lowercase on.A list of 500 email addresses, 37 of them repeated copies463 addresses, each where it first appeared. Turn on Ignore uppercase/lowercase if the same address was typed with different capitals.
Frequently asked questions
Two lines look the same but both survived — why?
Something invisible differs inside them: a non-breaking space, a curly apostrophe against a straight one, a look-alike letter from another alphabet, or an accent stored differently. Trim spaces only removes whitespace at the start and end of a line. If the only difference is capital letters, turn on Ignore uppercase/lowercase.
Does it keep the original order?
Yes. The first time a line appears it stays in place, and later repeats are dropped, so a list you had already arranged stays arranged. To sort it as well, use the Sort lists tool afterwards.
Which spelling is kept when case is ignored?
The first one in the list. With Banana, banana and BANANA in that order, the result keeps Banana.
How do I find the lines two lists have in common?
Use the List compare tool: it shows the lines in both lists, only in the first and only in the second.