In this article, let’s look at the different ways to find duplicates in an array in JavaScript. We’re going to be looking at different pre-defined JavaScript methods and finally some manual methods of finding the duplicates (usually using loops).
We’ll be looking at:
1. Finding if an array contains duplicates.
2. Getting an array of the duplicate elements and
3. Filtering out the duplicates to get an array of unique elements.
Using the filter and indexOf methods
One of the easiest and most efficient ways to filter an array and remove its duplicates is by using the ‘filter’ method. Combine ‘filter’ with ‘indexOf’, and you’re golden.
Let’s look at the different ways of using this combination in this section.
Remove the duplicates using the filter method
As you know, the ‘filter’ method can be used to filter an array based on a given condition. This method will return a new array, which can be re-assigned to the original array (to modify it) or to a new variable.
This method iterates through the entire array, and for every iteration, you get access to the current element and its index (optional). You can save these values in temporary variables (in our example, we’ve used a’ for the current value and ‘i’ for the current index).
Then, you can set the condition which would be checked for every iteration.
In our example, we’re checking if the index of the ‘a’ element is the same as ‘i’ using the indexOf method. This method returns the index of the first occurrence of the given element.
So, if the indices are the same, then that means it’s the first (or maybe the only occurrence) of the element in the array. If not, then there’s a duplicate and that duplicate element will not be considered in the filter.
At the end, you’ll end up with just the unique elements, and the duplicated values will be ignored, as shown below:
let arr = [1,2,1,4,3,2,5,3,6,4]; arr = arr.filter((a,i) => arr.indexOf(a) == i); console.log(arr); //[1, 2, 4, 3, 5, 6]
To find the duplicates using the filter method
You can modify the filter condition a little to get an array of the duplicates instead. Just look for the instances where the first index (first occurrence) of ‘a’ (current element) is not the same as the current index, which means we’re dealing with duplicates, as shown below:
arr = arr.filter((a,i) => arr.indexOf(a) !== i); //[1, 2, 3, 4]
Using the set object and ‘has’ method
The ‘set’ object is a great way to filter our array of duplicates. Why? Well, sets are unique, so they can’t accept duplicates by default.
By converting your array into a set, you’re automatically removing its duplicates. In this section, let’s look at the different ways of using ‘sets’ to find/remove duplicates in an array.
Remove duplicates using a set
The simplest way to remove the duplicates in your array is by first converting it into a set by using the new Set(arr) syntax.
Since sets can only have unique values, when you convert your array into a set, the duplicates are automatically removed.
Then, use the Array.from() method to convert this unique set back into an array and assign this array to a new variable (or the same one if you’d like to modify the original array), as shown below:
let arr = [1,2,1,4,3,2,5,3,6,4]; let uniqueArr = Array.from(new Set(arr)); //remove duplicates console.log(uniqueArr); [1, 2, 4, 3, 5, 6]
Once done, you’ll be left with only the unique values of the original array.
Check if an array has duplicates using a set
Once you convert an array into a set and convert it back into a unique array as we saw in the previous section, you can compare this newly created array to the original array to check if our array had any duplicates.
The simplest way to do this comparison is by using the ‘length’ property. If both arrays have the same length (same number of elements), your array does not have any duplicates, as shown below:
//find if original array had duplicates
if(arr.length === uniqueArr.length) {
console.log('The array does not have any duplicates');
}
else {
console.log('The array has duplicates');
}
Find the duplicates using the set object methods using the ‘has’ method
So far, we’ve looked at filtering an array of duplicates using sets and finding if an array did contain duplicates.
What if you want to find out what the duplicates are?
To do this,
1. You need to create an empty ‘duplicates’ array as usual.
2. Also, convert the original array into a set and keep it as it is (don’t convert it back into an array). We’re going to use this array to perform tests down the line.
3. Then, loop through the original array. For every iteration, use the ‘has’ method to check if the current element already exists in the set.
4. If it does, delete it from the set using the ‘delete’ method, as shown below:
let uniqueSet = new Set(arr); //got the unique values already
let duplicates = [];
//now filter the array, and remove all the unique values from the set so only the duplicates remain
arr.forEach((a) => {
if(uniqueSet.has(a)) {
uniqueSet.delete(a);
}
else {
duplicates.push(a);
}
});
console.log(duplicates); //[1, 2, 3, 4]
Then,
5. If the value does not exist in the set, that means it was deleted in a previous iteration, and we’ve encountered a duplicate.
6. Push this value to the ‘duplicates’ array. As usual, you can check if the value already exists in the ‘duplicates’ array before pushing as well.
That’s how you find the duplicates in an array using the ‘set’ object and the ‘has’ and ‘delete’ methods.
Find the duplicates and get a unique array in one go
So far, we’ve seen multiple ways of manipulating or using sets to find/remove duplicates from a given array.
Let’s combine what we’ve learned so far into one code snippet.
Look at the snippet below:
let set = new Set();
let duplicates = [];
arr.forEach((a) => {
if(set.has(a)) {
//then, it's a duplicate value
duplicates.push(a);
}
else {
set.add(a);
}
});
console.log(duplicates); //[1, 2, 3, 4]
console.log(Array.from(set)); //[1, 2, 4, 3, 5, 6]
This is what we’ve done above:
1. We’ve created an empty set and array. The empty set will store the unique values and the duplicates array will be used to store the values that were duplicated.
2. Loop through the original array. For every iteration, check if the current array element is already in the set.
3. If it is, then we’ve encountered a duplicate, so push this value into the ‘duplicates’ array. Alternatively, you can check whether the value already exists in the ‘duplicates’ array as well (repeated 3+ times) before you push it.
4. If it does not exist in the set yet, add the value to it using the ‘add’ method.
That’s it!
Using the ‘some’ method to find if the array has duplicates
The ‘some’ method can be used to return ‘true’ or ‘false’ values based on a given condition.
Just like the ‘filter’ method, the ‘some’ method also iterates through the given array, and for every iteration, you can check for a condition.
If this condition returns false for even 1 (or more) iteration, the entire ‘some’ method will return true, and false otherwise.
Look at the example below:
let arr = [1,2,1,4,3,2,5,3,6,4];
let duplicates = arr.some((a,i) => {
return arr.indexOf(a) !== i
});
if(duplicates) {
console.log('The array has duplicates')
}
else {
console.log('The array has no duplicates');
}
As you can see above, we’ve given the same condition we used in the ‘filter’ method. For every iteration, we’re checking if the index of the ‘a’ element is not the same as the current index ‘i’.
If this condition returns true even once, then that means our array has duplicates. Finally, we’re using an ‘if else’ statement to print our findings.
Unfortunately, this method can only be used to find if our array has duplicates. To remove the duplicates, or find what the duplicates are, you need to go for the other methods mentioned in this tutorial.
Using objects and keys
You can also use objects and their property-value pairs to find if your array has duplicates.
Look at the code snippet below:
let obj = {};
let duplicates = [];
arr.forEach((a) => {
if(!obj[a]) {
obj[a] = true; //create a new property value pair for the element
}
else {
//already created an entry so duplicate
duplicates.push(a);
}
});
console.log(duplicates); //[1, 2, 3, 4]
This is what we’ve in the snippet above:
1. To start with, we’ve created an empty object ‘obj’ and an empty array ‘duplicates’.
2. Then, we’re running a forEach loop on the original array. For every iteration of the loop, we’re assigning the current array element to the temporary variable ‘a’.
3. Then, for every iteration, we’re using an ‘if else’ statement to filter out our duplicates.
4. If obj[a] is false, as in, if ‘a’ is not yet a property of the ‘obj’ object, then that means we’re just now encountering this element, so let’s create a property ‘a’ in the object and assign it a value ‘true’ for future perusal.
5. If not, then that means this property was already created in the object. That is, we’ve already encountered the array element pertaining to this property. We’re dealing with a duplicate here.
6. So, push this element to the ‘duplicates’ array.
7. Additionally, you can check if the ‘duplicates’ array already has this element (in case the element was duplicated more than 2 times) by using the ‘includes’ method. This way, we’ll only get one occurrence of every duplicated value in the ‘duplicates’ array.
Manually find the duplicates using a nested for loop
You can also manually remove the duplicates (to get an array with just unique elements) by using a nested ‘for’ loop.
Look at the code snippet below:
let arr = [1,2,1,4,3,2,5,3,6,4];
let isDuplicate = false;
for(let i = 0; i < arr.length; i++) {
for(let j = i+1; j < arr.length; j++) {
if(arr[i] === arr[j]) {
isDuplicate = true;
break;
}
}
if(isDuplicate === true) break;
}
This is what we’ve done above:
1. To start with, we’ve created an ‘isDuplicate’ value and assigned it a starting value of ‘false’.
2. Then, we’ve created a nested ‘for’ loop. The outer loop runs from 0 till length-1, where ‘length’ is the number of elements in the original array.
3. The inner loop runs from i+1 to length-1. This way, the current element can be compared to the very next element (i+1).
4. If arr[i] and arr[j] are the same in any instance (which means we’ve encountered a duplicate), let’s make isDuplicate ‘true’.
5. Let’s also break away from the inner loop by using the ‘break’ statement.
6. If isDuplicate is true, then we’re done with the check. So, before the next iteration of the outer loop happens, check if the value is true, and if it is, then break out of the outer loop as well and print the results as you want.
Find the duplicates
If you’d like to find the duplicate values by using a nested loop, then modify the lines of code inside the inner ‘for’ loop a little.
Look at the snippet below:
let duplicates = [];
for(let i = 0; i < arr.length; i++) {
for(let j = i+1; j < arr.length; j++) {
if(arr[i] === arr[j]) {
if(!duplicates.includes(arr[i])) {
duplicates.push(arr[i]);
}
}
}
}
console.log(duplicates); //[1, 2, 4, 3]
As you can see, for every iteration of the inner loop, we’re checking if arr[i] and arr[j] are the same, as usual.
If they are, and we’re sure we’ve encountered a duplicate value, then we’re using the ‘includes’ method to check if the ‘duplicates’ array already has the arr[i] value. If it doesn’t, we’re storing this duplicate value in the array.
This way, we make sure that, even if a value is duplicated more than twice, only one entry of it makes it to the ‘duplicates’ value.
Finally, we can print the ‘duplicates’ array to display the list of the duplicated values. It’s as simple as that.
Manually remove the duplicates using loops
You can instead decide to remove the duplicate values from your array.
To do that, you’ll have to create a new array (uniqueArr) where the unique values will be stored.
For every iteration of the ‘for’ loop, check if the value at arr[i] already exists in ‘uniqueArr’. If it doesn’t, push it to ‘uniqueArr’.
This way, only the unique values will be stored, and you can later print it out.
let uniqueArr = [];
for(let i = 0; i < arr.length; i++) {
if(!uniqueArr.includes(arr[i])) {
uniqueArr.push(arr[i]);
}
}
console.log(uniqueArr); //[1, 2, 4, 3, 5, 6]
arr.forEach(a => {
if(!uniqueArr.includes(a)) {
uniqueArr.push(a);
}
});
Sort and use a ‘for’ loop to find duplicates
Another easy way to find the duplicates in an array is by using the ‘sort’ method. Apply this method on your array to sort your values in the ascending order.
Then, loop through this sorted array. If your array has duplicates, the current value and the one right next to it should be the same at some point. Check if it is.
If it is, and if the value does not already exist in the ‘duplicates’ array (to make sure we don’t store the same value twice), push it to the ‘duplicates’ array, as shown below:
let arr = [1,2,1,4,3,2,5,3,6,4];
arr.sort();
console.log(arr); //[1, 1, 2, 2, 3, 3, 4, 4, 5, 6]
let duplicates = [];
for(let i = 0; i < arr.length; i++) {
if(arr[i] === arr[i+1] && !duplicates.includes(arr[i])) {
duplicates.push(arr[i]);
}
}
console.log(duplicates); //[1, 2, 3, 4]
There you go! We’ve looked at various ways on how to find duplicates in an array in JavaScript.
We looked at using the ‘filter’ method, ‘set’ objects, simple objects with keys and finally, various ways of manually finding/removing the duplicates using ‘for’ loops.