Check If a List has Duplicate Elements

Lists are the most used data structures in Python. While programming, you may land into a situation where you will need a list containing only unique elements or you want to check if a list has duplicate elements. In this article, we will look at different ways to check if a list has duplicate elements in it.

Check if a list has duplicate Elements using Sets

We know that sets in Python contain only unique elements. We can use this property of sets to check if a list has duplicate elements or not. 

For this, we will create a set from the elements of the list. After that, we will check the size of the list and the set. If the size of both the objects are equal, it will confirm that the list has no duplicate elements. If the size of the set is greater than the list, it will mean that the list contains duplicate elements.  We can understand this from the following example.

def check_duplicate(l):
    mySet = set(l)
    if len(mySet) == len(l):
        print("List has no duplicate elements.")
    else:
        print("The list contains duplicate elements")


list1 = [1, 2, 3, 4, 5, 6, 7]
print("List1 is:", list1)
check_duplicate(list1)
list2 = [1, 2, 1, 2, 4, 6, 7]
print("List2 is:", list2)
check_duplicate(list2)

Output:

List1 is: [1, 2, 3, 4, 5, 6, 7]
List has no duplicate elements.
List2 is: [1, 2, 1, 2, 4, 6, 7]
The list contains duplicate elements

In the above approach, we need to create a set from all the elements of the list. After that, we also check the size of the set and the list. These operations are very costly. 

Instead of using this approach, we can search only for the first duplicate element. To do this, we will start from the first element of the list and will keep adding them to the set. Before adding the elements to the set, we will check if the element is already present in the set or not. If yes, the list contains duplicate elements. If we are able to add each element of the list to the set, the list does not contain any duplicate element. This can be understood from the following example.

def check_duplicate(l):
    visited = set()
    has_duplicate = False
    for element in l:
        if element in visited:
            print("The list contains duplicate elements.")
            has_duplicate = True
            break
        else:
            visited.add(element)
    if not has_duplicate:
        print("List has no duplicate elements.")


list1 = [1, 2, 3, 4, 5, 6, 7]
print("List1 is:", list1)
check_duplicate(list1)
list2 = [1, 2, 1, 2, 4, 6, 7]
print("List2 is:", list2)
check_duplicate(list2)

Output:

List1 is: [1, 2, 3, 4, 5, 6, 7]
List has no duplicate elements.
List2 is: [1, 2, 1, 2, 4, 6, 7]
The list contains duplicate elements.

Check if a list has duplicate elements using the count() method

To check if a list has only unique elements, we can also count the occurrence of the different elements in the list. For this, we will use the count() method. The count() method, when invoked on a list, takes the element as input argument and returns the number of times the element is present in the list.  

For checking if the list contains duplicate elements, we will count the frequency of each element. At the same time, we will also maintain a list of visited elements so that we don’t have to count the occurrences of the visited elements. Once the count of any element is found to be greater than one, it will prove that the list has duplicate elements. We can implement this as follows.

def check_duplicate(l):
    visited = set()
    has_duplicate = False
    for element in l:
        if element in visited:
            pass
        elif l.count(element) == 1:
            visited.add(element)
        elif l.count(element) > 1:
            has_duplicate = True
            print("The list contains duplicate elements.")
            break
    if not has_duplicate:
        print("List has no duplicate elements.")


list1 = [1, 2, 3, 4, 5, 6, 7, 8]
print("List1 is:", list1)
check_duplicate(list1)
list2 = [1, 2, 1, 2, 4, 6, 7, 8]
print("List2 is:", list2)
check_duplicate(list2)

Output:

List1 is: [1, 2, 3, 4, 5, 6, 7, 8]
List has no duplicate elements.
List2 is: [1, 2, 1, 2, 4, 6, 7, 8]
The list contains duplicate elements.

 Check if a list has duplicate elements using the counter() method

We can also use the counter() method to check if a list has only unique elements or not. The counter() method. The counter() method takes an iterable object as an input and returns a python dictionary in which the keys consist of the elements of the iterable object and the values associated with the keys are the frequency of the elements. After getting the frequency of each element of the list using the counter() method, we can check if the frequency of any element is greater than one or not. If yes, the list contains duplicate elements. Otherwise not.

from collections import Counter


def check_duplicate(l):
    counter = Counter(l)
    has_duplicate = False
    frequencies = counter.values()
    for i in frequencies:
        if i > 1:
            has_duplicate = True
            print("The list contains duplicate elements.")
            break
    if not has_duplicate:
        print("List has no duplicate elements.")


list1 = [1, 2, 3, 4, 5, 6, 7, 8]
print("List1 is:", list1)
check_duplicate(list1)
list2 = [1, 2, 1, 2, 4, 6, 7, 8]
print("List2 is:", list2)
check_duplicate(list2)

Output:

List1 is: [1, 2, 3, 4, 5, 6, 7, 8]
List has no duplicate elements.
List2 is: [1, 2, 1, 2, 4, 6, 7, 8]
The list contains duplicate elements.

Conclusion

In this article, we have discussed four ways to check if a list has only unique elements or not. We have used sets, count() and counter() methods to implement our approaches. To learn more about lists, you can read this article on list comprehension.