Duplicate Line Remover
Strip duplicate lines from text. Case toggle, trim-before-compare, optional sort, stats. Free, offline, client-side, instant, secure.
Paste text on the left, get unique lines on the right. Toggle case sensitivity, ignore whitespace, or sort alphabetically. Order is preserved by default.
How to Use Duplicate Line Remover
- Paste text into the left textarea. Each newline separates a line.
- Pick options. Case sensitive on means "Apple" and "apple" stay separate. Trim means leading/trailing whitespace is ignored when matching (the kept line still keeps its original whitespace). Sort outputs alphabetically instead of preserving order.
- Read the result on the right - updates live as you type.
- Check the stats line: input lines, unique lines, duplicates removed (with percentage), and longest line length.
- Copy or download the result. Filename is
unique-lines.txt. Ctrl/Cmd + Enter focuses the output for fast selection.
Frequently Asked Questions
How is duplicate detection done?
The input is split on newline characters into individual lines. A JavaScript Set tracks which lines have already been seen. For each line, the comparison key is the raw line (or the lowercased / trimmed version, depending on the toggles); only the first occurrence is kept, and the original (un-modified) line is what gets written to the output.
What does “trim before compare” do?
When on, leading and trailing whitespace is stripped from each line before comparing. So " alpha " and "alpha" collide and only the first one is kept. The output retains the original whitespace of the first occurrence – the trim only affects the comparison key, not the kept text.
Does this preserve the original line order?
By default yes – the first occurrence of each unique line is kept and the rest are removed, so the relative ordering of unique lines matches the input. If you enable “Sort output alphabetically”, that ordering is replaced with a locale-aware string sort.
What’s the difference between case sensitive and insensitive modes?
Case sensitive (the default) treats “Apple”, “apple”, and “APPLE” as three different lines – all three will appear in the output. Case insensitive treats them as the same key – only the first occurrence appears. Useful for cleaning lists of emails, usernames, or filenames where capitalization should not matter.
What about empty lines?
Empty lines are treated as a regular line with empty content. If your input has multiple blank lines in a row, only the first one is kept (the rest are duplicates of empty string). If you need to preserve blank lines, remove them from the input first, or add them back after deduplication.
How fast is it on large input?
Linear in the number of lines (O(N) using a Set). 100,000 lines run in well under a second on a modern machine. A 100 ms debounce avoids re-running on every keystroke when you paste large blocks.
Is anything sent to a server?
No. The page loads three static files (HTML, CSS, JS) and then runs entirely in your browser. You can disconnect from the internet after the page loads and the tool keeps working. No analytics, no tracking, no cookies.
Is this tool free?
Yes – free, unlimited, no signup, no watermark, no usage cap. Use the output in any context. Attribution to is appreciated but not required.