Extract Unique List Items
Dedupe a list, find duplicates, or keep only singles - newline/comma/tab/pipe input, 5 modes, 4 sort orders. Free, offline, client-side, instant, secure.
Dedupe a list, pull out only duplicates, keep only singletons, or grab the first or last occurrence of each item. Six input separators (newline / comma / tab / pipe / semicolon / space), four sort orders, and live preview.
How to Use Extract Unique List Items
- Paste your list in the input. Output updates live (200 ms debounce).
- Pick the input separator. Newline is the most common; switch to comma for CSV-style lines, tab for TSV, pipe for log lines.
- Pick a mode.
- All unique items - one copy per distinct item (first seen). Removes duplicates.
- Only duplicates - only items that appear more than once.
- Only singles - only items that appear exactly once.
- First occurrence - same as "all unique" (one copy per item, first position kept).
- Last occurrence - one copy per item, picking the last position instead of the first.
- Pick a sort order. Original keeps first-appearance order. Alphabetical sorts A→Z or Z→A using your locale. Frequency puts the most-repeated items first (ties broken alphabetically).
- Toggle the options. "Case sensitive" makes "Apple" and "apple" different items. "Trim whitespace" strips leading/trailing spaces before comparing (so " apple" and "apple" are the same).
- Read the stats line. Input item count, distinct item count, returned item count, duplicates removed (if any), max frequency (when there are repeats).
- Copy or download. Ctrl/Cmd + Enter copies. Download saves
unique-items.txt.
Frequently Asked Questions
Is my data secure when extracting unique items?
Yes, absolutely. All extraction and deduplication is performed entirely in your browser using client-side JavaScript. Your data never leaves your device, is never uploaded to any server, and is not stored anywhere. This tool works completely offline after the page loads.
What’s the difference between extraction modes?
All unique items removes duplicates and returns one copy of each item. Only duplicates returns items that appeared multiple times. Only singles returns items that appeared exactly once. First / Last occurrence keep the first or last appearance of each item while preserving original order.
How does case sensitivity work?
When case sensitive is disabled (default), “Apple” and “apple” are treated as the same item – only the first-seen casing is kept. When enabled, they’re treated as different items and both will be kept. The toggle affects both deduplication and alphabetical sorting.
What does “Sort by frequency” do?
Sort by frequency orders items by how many times they appeared in the original list, with the most common items first. Items with the same frequency are then sorted alphabetically. Combined with “Only duplicates” mode, this gives you a “top duplicates” report.
Can I preserve the original order?
Yes – select “Original order” in the Sort Order dropdown to keep items in the same sequence they appeared in your input. The position used is the first occurrence’s position by default; in “Last occurrence” mode, the order follows where the last copy was.
What’s the maximum list size I can process?
The tool handles tens of thousands of items efficiently using Set and Map data structures. Hundreds of thousands work but the 200 ms input debounce starts feeling sluggish. For 1M+ items, paste once and let it finish – the algorithm itself is O(n).
Can I use different separators for input and output?
Yes. By default the output separator follows the input separator, but you can change them independently. Common combo: read comma-separated input from Excel, output as newline-separated for a Markdown bullet list.
What does “Trim whitespace” do?
When enabled, this option removes leading and trailing spaces from each item before comparison. This prevents items like “apple” and ” apple ” from being treated as different. Leave it on unless you specifically need to distinguish whitespace-padded entries.
Is this tool free?
Yes – free, unlimited, no signup, no watermark. The extracted list is yours to use anywhere. Attribution to is appreciated but not required.