Python Polars group_by Aggregation and Sorting

Polars group_by lets you summarize data by category and then order the results using the expression API. This article covers the core patterns for group_by aggregation and sorting, including multiple aggregations, column naming, sorting within groups, lazy evaluation, and null handling.

Basic group_by and Aggregation

The fundamental operation is to call group_by on one or more columns and pass aggregation expressions to agg. For example, given a DataFrame of sales records, you can compute total sales per region:

import polars as pl

df = pl.DataFrame({
    "region": ["North", "South", "North", "East", "South", "East"],
    "sales": [100, 200, 150, 300, 250, 400],
})

result = df.group_by("region").agg(pl.col("sales").sum())
print(result)

The result is a DataFrame with one row per region and a column named sales containing the sum. If you want a more descriptive column name, use alias:

result = df.group_by("region").agg(pl.col("sales").sum().alias("total_sales"))

Sorting Aggregated Results

After aggregation, sort the new DataFrame directly:

result = df.group_by("region").agg(pl.col("sales").sum().alias("total_sales"))
sorted_result = result.sort("total_sales", descending=True)
print(sorted_result)

This sorts the aggregated rows by total_sales in descending order. You can also sort by multiple columns, for example by region and then by total sales:

sorted_result = result.sort(["region", "total_sales"])

Sorting after aggregation is the most common pattern because the aggregated values do not exist until agg runs.

Multiple Aggregations and Column Naming

Pass a list of expressions to agg to produce multiple summary statistics per group:

result = df.group_by("region").agg([
    pl.col("sales").sum().alias("total_sales"),
    pl.col("sales").mean().alias("avg_sales"),
    pl.col("sales").count().alias("num_sales"),
])
print(result)

Use alias to give each statistic a distinct name, especially when several aggregations use the same source column. You can then sort by any of these columns:

result.sort("avg_sales", descending=True)

Common aggregations include:

Sorting Values Within Groups

Sometimes the aggregation itself depends on row order inside each group. Sorting the column expression before aggregating can support patterns such as selecting the largest value per group:

result = df.group_by("region").agg(
    pl.col("sales").sort().last().alias("largest_sale")
)

This sorts the sales values in ascending order within each group and selects the last, largest value. For the top N values per group, combine sort and head:

result = df.group_by("region").agg(
    pl.col("sales").sort(descending=True).head(2).alias("top_two_sales")
)

The result is a list column containing the two largest sales for each region. Sorting within groups happens inside the agg expression; sorting the final aggregated DataFrame happens after agg.

Performance: Lazy Evaluation and the Expression API

With the lazy API, Polars can optimize the entire query before executing it. Build the query with lazy() and call collect() at the end:

lazy_result = (
    df.lazy()
    .group_by("region")
    .agg(pl.col("sales").sum().alias("total_sales"))
    .sort("total_sales", descending=True)
    .collect()
)

This can reduce intermediate materialization and let Polars optimize the plan as a whole. Whether it is faster depends on the data and operations, but it is especially useful for larger pipelines that combine filtering, joins, and aggregation.

Another practical point is to avoid separate group_by calls when you need several statistics. Combine them in one agg rather than running multiple aggregations and joining the results. This reduces the number of passes over the data.

Handling Nulls and Missing Groups

group_by includes nulls as their own group by default. For example:

df = pl.DataFrame({"cat": ["A", None, "A"], "val": [1, 2, 3]})
result = df.group_by("cat").agg(pl.col("val").sum())
print(result)

This produces one group for "A" and one for null. If you want to exclude null groups, filter them after aggregation:

result = df.group_by("cat").agg(pl.col("val").sum()).filter(pl.col("cat").is_not_null())

When sorting, use nulls_last=True to keep missing groups at the bottom:

result.sort("total_sales", nulls_last=True)

Common Mistakes with group_by and sort

A frequent error is adding more than one aggregation on the same source column without alias. Duplicate output column names make the result hard to use, so alias each statistic when you need multiple aggregations.

Another mistake is sorting the original DataFrame before group_by when the goal is to order the aggregated groups. Sorting before aggregation changes row order inside groups, but it does not set the order of the groups in the result. To order groups, sort after agg.

Finally, note that group_by does not preserve the original row order. The output is ordered by the grouping key by default. If you need a custom order, apply an explicit sort after aggregation. With multiple grouping columns, the default order is lexicographic by the group keys.

For complex pipelines, using the lazy API with sort after group_by keeps the intent clear and lets Polars optimize the execution plan. The expression API is consistent between eager and lazy modes, so you can switch without changing the aggregation logic.