Text File Generator
Duplicates creep into text from everywhere. Copied lists, merged exports, tags typed twice, a keyword you have used four times in one paragraph without noticing. This tool strips the repeats out and leaves one of each, and the mode you pick decides what counts as a repeat in the first place.
Pick the mode before anything else
Words across the whole text finds every repeated word anywhere in what you pasted and flattens the result into a single run of words. It is the right choice for building a vocabulary list or a tag set out of a block of writing, and the wrong choice if you wanted to keep your paragraphs, because line breaks do not survive it.
Words within each line does the same job line by line, so a line reading "red blue red green" becomes "red blue green" while every other line keeps its own words and its own position. Your line structure stays exactly as it was.
Lines mode compares whole lines rather than individual words, which is what you want for a list of emails, URLs, or names. Comma separated values does the same for a single string of values, and it tidies up the spacing around the commas while it works.
Keeping the first or the last one
When a value appears more than once, one of them survives, and which one is not always arbitrary. Keeping the first occurrence preserves the order things were originally entered, which suits a list where earlier entries are more authoritative.
Keeping the last one matters when your text is chronological. In a log of updates where the newest version of a record sits at the bottom, keeping the first would leave you with the stale copy of every entry. The output order stays the same either way, only the choice of which duplicate survives changes.
Making the match looser
Ignore case is on by default, because "Apple" and "apple" are almost always the same thing when you are deduplicating. Turn it off for anything case sensitive such as codes, identifiers, or passwords, where two values differing only in case really are different values.
Ignore punctuation goes a step further and compares words with the punctuation stripped out, so "hi!" and "hi" match. The surviving copy keeps its punctuation exactly as you typed it, since only the comparison ignores it, not the output.