Delete Repeating List Items
Remove duplicates from any list. 3 modes (consecutive, all, run-length collapse to N). 6 separators. Trim before compare. Free, offline, client-side, instant, secure.
Three dedup modes: All duplicates (only first occurrence kept), Consecutive only (adjacent dups removed, distant ones kept), and the new Run-length collapse to N (e.g. N=2 keeps up to 2 consecutive copies). Works with newline, comma, semicolon, pipe, or space separators.
How to Use Delete Repeating List Items
- Paste your list. Pick the separator that matches your input (the original only allowed 3, now 6).
- Pick dedup mode: All duplicates (default - first occurrence wins), Consecutive only (adjacent dups removed, distant occurrences preserved), or Run-length collapse to N (e.g.
[A,A,A,A,B]with N=2 →[A,A,B]). - Tick case-sensitive (default ON), skip empty items (default ON), or trim before compare (off by default) for fine control.
- Live preview after 200 ms. Ctrl+Enter recalculates manually.
- Stats: original count / after-dedup count / removed / items-that-had-duplicates (each counted once regardless of how many dups it had).
- The output preserves your input separator (commas in → commas out - the original silently swapped to newlines).
Frequently Asked Questions
Consecutive vs All vs Run-length – when to use which?
All duplicates (default): keep only the first occurrence of each unique item. [A, B, A, C, B, A] → [A, B, C]. Use for general deduplication.
How does the “Items with dups” stat differ from “Removed”?
Removed = total items deleted (raw count). Items with dups = unique items that had at least one duplicate (each counted once). Example: [A, A, A, B, B, C] → All-mode result [A, B, C]. Removed = 3 (two A’s + one B). Items with dups = 2 (A and B). The Items-with-dups stat shows “how varied were the duplicates”, not the total count.
What’s “Trim before compare”?
Strips leading/trailing whitespace from each item before comparing for duplicates. "apple" and " apple " become identical after trim, so they’re treated as duplicates. The trimmed key is used for COMPARISON only – the kept item retains its original whitespace. So output might show "applen cherry" (untrimmed) even though comparison was case-and-space normalized.
How does case-sensitive work?
Default is ON (the HTML’s checked attribute). With ON: "Apple" and "apple" are different items. With OFF: they’re treated as duplicates (compared in lowercase). Useful when your list might have inconsistent capitalization from copy-paste or merged sources. The original FAQ said “When disabled (default)” which contradicted the UI default.
How is “Items with dups” useful?
Shows the diversity of your duplicate problem. If you have 1000 items, removed 800, and Items-with-dups is just 5 – your list is dominated by a few items with massive runs. If removed is 800 and Items-with-dups is 200 – your dups are spread across many items. Helps decide whether to investigate a few hot offenders (low Items-with-dups, high removed) or apply broader cleaning.
What’s the relationship between Consecutive mode and Run-length collapse?
Consecutive mode is run-length collapse to N=1 (keep only the first of each run). Run-length collapse with N=1 produces identical output. With N=2, you can have up to 2 consecutive copies – useful when “AA” is meaningful (like double letters in words) but “AAAA” is excessive. Higher N values become equivalent to no-op as N approaches your input length.
Can I do partial / fuzzy duplicate matching?
Not directly. The tool does strict equality after the configured normalization (trim + case). For fuzzy matching (“apple” matches “aple”), use a Levenshtein-distance tool. For “contains” matching (extract items containing a substring), use the sibling “Delete List Items” tool. For phonetic matching, a Soundex/Metaphone tool. Each is a different problem.
Is my data secure?
Yes. All processing happens in your browser using Set-based O(n) algorithms. Your list never leaves your device.