Java HashSet: Remove Duplicates from a List

Removing duplicates from a List in Java is straightforward with HashSet: pass the list to the HashSet constructor, then convert the resulting set back to a List if you need a list result. The set's equality rules determine which values are treated as duplicates, so the element type must honor the equals() and hashCode() contract.

How HashSet Removes Duplicates

A HashSet is backed by a HashMap internally. When you call add(element), the set computes the element's hash code, locates the corresponding bucket, and checks whether an equal element already exists using the equals() method. If an equal element is found, the new element is discarded and add returns false. Otherwise, the element is inserted.

This means deduplication is not based on identity or reference equality. Two distinct objects that are equals() to each other are treated as duplicates. For String, Integer, and other standard library types, equals() already provides value-based comparison, so deduplication works as expected out of the box.

The Basic Deduplication Pattern

The simplest way to remove duplicates from a List is to pass it to the HashSet constructor:

List<String> input = Arrays.asList("java", "python", "java", "go", "python");
Set<String> unique = new HashSet<>(input);
List<String> result = new ArrayList<>(unique);

The HashSet constructor iterates over the input collection and adds each element. Duplicates are silently ignored. Converting the set back to a List gives you a collection without duplicates, but the order is not guaranteed to match the original input order.

If you only need to check membership or iterate over unique values, you can keep the Set directly instead of converting back to a List.

Preserving Order with LinkedHashSet

HashSet does not guarantee iteration order. If the original order of the first occurrence matters, use LinkedHashSet instead:

List<String> input = Arrays.asList("java", "python", "java", "go", "python");
Set<String> unique = new LinkedHashSet<>(input);
List<String> result = new ArrayList<>(unique);
// result: [java, python, go]

LinkedHashSet maintains a doubly linked list of entries alongside the hash table, so iteration follows insertion order. The deduplication logic is identical to HashSet; only the ordering behavior differs. The memory overhead is slightly higher because of the linked list structure, but for typical list sizes this is negligible.

Removing Duplicates from Custom Objects

For custom classes, deduplication only works correctly if equals() and hashCode() are properly overridden. Consider a User class:

public class User {
    private final String email;
    private final String name;

    public User(String email, String name) {
        this.email = email;
        this.name = name;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (!(o instanceof User)) return false;
        User other = (User) o;
        return email.equals(other.email);
    }

    @Override
    public int hashCode() {
        return email.hashCode();
    }
}

With this implementation, two User objects with the same email are considered equal, so a HashSet will keep only the first one. If equals() and hashCode() are not overridden, the set falls back to Object's reference-based equality, and two separate objects with identical field values will both be kept.

The contract between equals() and hashCode() is critical: if two objects are equal, they must have the same hash code. Violating this breaks the set's bucket lookup and causes duplicates to appear.

Performance and Memory Considerations

The time complexity of adding an element to a HashSet is O(1) on average, assuming a well-distributed hash function. The full deduplication of a list of n elements is therefore O(n) on average. This is significantly faster than the O(n²) nested-loop comparison approach.

The memory cost includes the hash table itself, which uses more space than a plain ArrayList holding the same elements. The default initial capacity is 16, and the set resizes when the load factor of 0.75 is reached. If you know the approximate number of unique elements in advance, you can pass an initial capacity to avoid repeated resizing:

Set<String> unique = new HashSet<>(expectedUniqueCount);

For very large collections, the hash function quality matters. Poorly distributed hash codes cause many collisions, degrading lookup to O(n) in the worst case and increasing memory usage.

Edge Cases: Nulls and Mutable Objects

HashSet permits one null element. Adding null twice results in a single entry; the set treats null like any other value for deduplication purposes. This is usually fine, but be aware that the resulting collection cannot contain more than one null.

Mutable objects in a set are a known hazard. If you add an object to a HashSet and then mutate it in a way that changes its hashCode(), the set can no longer find the object in its bucket. The element becomes orphaned, and duplicate entries can appear if the same logical value is added again. If you must store mutable objects, remove them from the set before mutation and re-add them afterward.

For the common case of deduplicating strings, numbers, or immutable value objects, HashSet is the right choice. For order-sensitive deduplication, LinkedHashSet provides the same behavior with insertion-order iteration. The only real requirement is that the element type honors the equals()/hashCode() contract.