notepad++ or excel - delete duplicate ad original rows - duplicates

Question

I have a single .txt with +200k rows.
In this file, i want to delete duplicates and original rows.
I have now:
text a
text a
text b
text c
text d
text d
text e
But i need a result like this
text b
text c
text e
Suggest?
i have tried normal "delete duplicate" procedure of excel and notepad++ but i obtain this
text a
text b
text c
text d
text e
and it not work fine for me
looking for discussion i find something like that but applicated to access.

Answer 1

Just using Notepad++, do a regular expression replace of ^(.*\R)\1+ with nothing. Ensure that "Wrap around" is selected and that ". matches newline" is not selected. This assumes that the duplicate lines are grouped together; sorted lines would be ideal.
The regular expression is as follows
^ Beginning of a line
( Start capture group
.* Zero or more of any character
\R An end of line
) End of capture group
\1+ One or more repeats of the capture group
If the last line of the file is part of a duplicated group but it does not have the end-of-line character(s) then the above regular expression will not find that last line. If the last three or more lines are identical, except that there is no final end-of-line character(s) then the above regular expression will remove most of the final group but leave one line. My recommendation is to ensure that the last line of the file has the end-of-line character(s) before using the regular expression. (Thanks to #Toto for pointing out this possible problem.)
Note that if the test for what is duplicated is more complex than exactly identical then the ideas in this Stack Overflow answer may be used.

Answer 2

Here is a way how you can achieve it using python. Hope it helps :)
# Importing Pandas to create DataFrame
import pandas as pd
# Creating Empty DataFrame and Storing it in variable df
df = pd.DataFrame()
#replce this with our requiremnt
# lt = ["text a", "text a", "text b", "text c", "text d", "text d", "text e"]
text_file = open(r"C:\Users\antch\Desktop\txt.txt", "r")
lt = text_file.readlines()
# print (lt)
# print (type(lt))
text_file.close()
df['cols'] = lt
#creating an empty column with count value set to 1
df['dummy_count'] = 1
# grouping values
df = df.groupby('cols').count()
# print(df)
df.reset_index(inplace = True)
df = df.loc[df["dummy_count"] == 1]
print(df["cols"])
the output of the above code: expected output