Using uniq command in Linux

Linux provides various tools to work with text files. The uniq is a command used to detects and deletes duplicate lines in a file. This can be very helpful to remove duplicate text in large text files.

In this article, you see the usage of uniq command along with some examples.

The syntax of the Linux uniq command

The syntax of using uniq command in Linux is given below-

uniq [Option] [Input [Output]]

You find the options that can be used with this command on its man page.

How to remove duplicate lines using uniq command

Suppose we have a file called file1.txt which contains some duplicate lines of text. You can see the content of this file in the given image –

Now to remove all repeated lines use the following command –

uniq file1.txt

You can see the output of this command in the given image –

How to display the number of times a line repeated

By using option -c you can display the number of times a line is repeated. Now execute the following command to display the count of line repetition.

uniq -c file1.txt

You can see the output in the given image –

How to display duplicate lines only

By using option -D with uniq command will print only duplicate lines of text. Use the following command –

uniq -D file1.txt

See the output in the image below –

Similarly, you can use option -u to print non-repetitive lines only.

uniq -u file1.txt

How to make uniq comparison insensitive

By default, uniq command comparison is case sensitive. You can make this command to perform the insensitive comparison by using the option -i. The complete command is given below-