Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

How to Use LINQ DistinctBy in C#

Learn how to use LINQ DistinctBy in C# to remove duplicate objects based on a specific property, with practical examples, custom comparers, and performance tradeoffs.

~6 min readC#
#LINQ#Distinct#DistinctBy#C#
Code snippet showing LINQ DistinctBy with a key selector that filters duplicate objects based on a single property

When you need to remove duplicate items from a collection, the standard LINQ Distinct method compares entire objects. If you have a list of Person objects and want to keep only the first occurrence of each Email, Distinct alone won't help unless you override Equals. The DistinctBy method, introduced in .NET 6, solves this directly: it accepts a key selector and returns objects that are unique according to that key. This article focuses on DistinctBy usage, its behavior, and when to choose it over alternatives.

Basic Syntax and First Example

DistinctBy is an extension method in the System.Linq namespace. It takes a Func<TSource, TKey> key selector and returns an IEnumerable<TSource> containing the first element from each group of objects that share the same key.

For example, given a list of orders:

csharp
public record Order(int Id, string Customer, decimal Amount, DateTime Date); List<Order> orders = new() { new(1, "Alice", 100m, new DateTime(2024, 1, 1)), new(2, "Bob", 50m, new DateTime(2024, 1, 2)), new(3, "Alice", 200m, new DateTime(2024, 1, 3)), new(4, "Carol", 75m, new DateTime(2024, 1, 4)) }; var distinctCustomers = orders.DistinctBy(o => o.Customer).ToList();

The result is the first order for Alice, the order for Bob, and the order for Carol—three orders total. The second order for Alice—with Id 3—is skipped because Alice already appeared.

The key selector can be any expression that returns a value used for equality comparison. The default equality comparer for the key type is used, so for strings it's ordinal, for integers it's numeric equality, and for custom types it uses the type's Equals implementation.

How Key Selection Works

The key selector determines what counts as a duplicate. If you need to compare by multiple properties, you can use an anonymous type or a tuple:

csharp
var uniqueByCustomerAndDate = orders .DistinctBy(o => new { o.Customer, o.Date }) .ToList();

Anonymous types in C# implement value-based equality, so two anonymous objects with the same property values are considered equal. This lets you combine fields without writing a custom comparer.

Alternatively, you can use a tuple, which also has value semantics:

csharp
var uniqueByCustomerAndDate = orders .DistinctBy(o => (o.Customer, o.Date)) .ToList();

Both approaches work, but anonymous types are often clearer when you have more than two fields.

Practical Example: Removing Duplicate Records

A common use case is cleaning up a data feed where duplicate records arrive with the same ID but different timestamps. You want the latest record for each ID. DistinctBy alone keeps the first, not the latest, so you need to combine it with sorting:

csharp
var latestBySensorId = sensorReadings .OrderByDescending(r => r.Timestamp) .DistinctBy(r => r.SensorId) .ToList();

This sorts the readings from newest to oldest, then DistinctBy keeps the first (newest) for each sensor ID. If you don't sort first, you'll keep the earliest occurrence in the original order, which may not be what you want.

This pattern is useful when processing event streams or log entries where the same logical entity appears multiple times.

Working with Custom Types and Comparers

When the key type is a custom class that doesn't override Equals, DistinctBy uses reference equality by default. This means two objects with identical field values won't be considered duplicates unless they are the same instance.

To control comparison, you can pass a custom IEqualityComparer<TKey> as the second argument:

csharp
public class CaseInsensitiveComparer : IEqualityComparer<string> { public bool Equals(string? x, string? y) => string.Equals(x, y, StringComparison.OrdinalIgnoreCase); public int GetHashCode(string obj) => obj.ToUpperInvariant().GetHashCode(); } var uniqueNames = people .DistinctBy(p => p.Name, new CaseInsensitiveComparer()) .ToList();

The comparer is also used for hashing, so you must implement GetHashCode consistently with Equals. If the hash codes differ for values that should be equal, DistinctBy may not work correctly.

DistinctBy vs Distinct vs GroupBy

Distinct compares whole objects. DistinctBy compares a selected key. GroupBy groups objects by key and lets you process each group. Here's a comparison:

MethodWhat it doesKey selectionReturns
DistinctRemoves duplicate whole objectsNoUnique objects
DistinctByRemoves objects where the key duplicatesYesFirst object per key
GroupByGroups objects by keyYesGroups of objects

DistinctBy is equivalent to GroupBy(key).Select(g => g.First()) but with a simpler syntax and potentially better performance because it doesn't build full groups.

When you need more than the first element per key—for example, the sum or count of each group—use GroupBy instead.

Performance and Memory Considerations

DistinctBy works by maintaining an internal HashSet<TKey> of seen keys. As it iterates through the source, it checks each key against the set and only yields the element if the key hasn't been seen. This gives an average time complexity of O(n) for typical cases, but with extra memory usage proportional to the number of unique keys.

If the key type has expensive hash computation or the collection is extremely large, consider the cost of hashing. For most business objects this is negligible. For very large streams where memory is a concern, you could implement a streaming approach that writes to disk, but that's rarely necessary.

Another point: DistinctBy is deferred—it doesn't evaluate the source until you iterate the result, for example by calling ToList(). This means you can chain it with other LINQ operators and defer execution.

Where DistinctBy Is Available

DistinctBy is part of .NET 6 and later. If you're working with .NET Framework or earlier versions of .NET Core, you won't have it built-in. In those cases, you can write your own extension method or use a third-party library. A simple implementation is:

csharp
public static IEnumerable<TSource> DistinctBy<TSource, TKey>( this IEnumerable<TSource> source, Func<TSource, TKey> keySelector) { var seen = new HashSet<TKey>(); foreach (var item in source) { if (seen.Add(keySelector(item))) yield return item; } }

This replicates the built-in method's default key comparison for older runtimes. It's a good learning exercise to understand how the method works internally.

Common Pitfalls

One common mistake is assuming DistinctBy keeps the last occurrence. It keeps the first. If you need the last, reverse the source or sort by a descending key before calling DistinctBy.

Another issue is using a mutable key. If the key object changes after it has been added to the internal hash set, the set may misbehave because the object's hash code changes. Avoid mutating key values while DistinctBy is enumerating the source.

Also be careful with nullable keys. DistinctBy treats null as a valid key, so all elements with a null key will be collapsed into one. If that's not intended, filter out nulls first.

When to Use DistinctBy in Production

DistinctBy is a clean, readable way to deduplicate data based on a property without writing a custom comparer for every type. Use it when:

  • You're on .NET 6 or later.
  • You need the first object per key.
  • The key is a simple property or a combination of properties.
  • You don't need to aggregate the group.

For older frameworks, write a custom extension method or use a compatibility package. The implementation is small and well-understood, so the maintenance burden is low.

In scenarios where you need to control equality precisely—like case-insensitive matching or cultural-specific comparisons—pass a custom comparer instead of relying on the default.

DistinctBy fits naturally into a pipeline of LINQ operations. For example, you can filter, project, sort, and then deduplicate in a single expression, making the data transformation intent explicit. That readability is often worth more than micro-optimizations.

Finally, remember that DistinctBy preserves order: the result contains elements in the same order they appeared in the source, with duplicates skipped. This can be important when the source order carries meaning, such as a chronological event log.

If you're deduplicating a large dataset that doesn't fit in memory, consider a streaming approach or a database-side query that picks one row per key instead of pulling everything into memory. DistinctBy is best suited for in-memory collections where the number of items is manageable.

Try matching browser tools

All tools

Small utilities that fit C# work — they run entirely in this browser tab, so nothing you paste leaves your device.

All C# guides

More in C#

All C# guides

The guides filed next to this one in C#, in library order.