Python Polars: select, filter, and with_columns Expressions
Learn how to use Polars' select, filter, and with_columns expressions to choose columns, filter rows, and add derived columns in DataFrame pipelines.

When working with Polars, select, filter, and with_columns form the core of many data transformations. They let you choose columns, filter rows, and add or modify columns in a composable way. This article explains how each method works, how to combine them in pipelines, and how eager and lazy evaluation affect their use.
Understanding Expressions in Polars
In Polars, an expression describes a computation over one or more columns. It can be reused, combined with other expressions, and optimized by the query engine. The select, filter, and with_columns methods all accept expressions, which is what makes them flexible.
For example, pl.col("age") is an expression for the age column. You can apply operations to it: pl.col("age").mean() or pl.col("age").alias("average_age"). Expressions can also be combined with operators like +, -, &, |, and >.
Selecting Columns with select
The select method returns a new DataFrame containing only the columns produced by the supplied expressions. The simplest use is passing column names as strings, but pl.col gives you more flexibility for building expressions.
import polars as pl
df = pl.DataFrame({
"name": ["Alice", "Bob", "Charlie"],
"age": [25, 30, 35],
"city": ["NYC", "LA", "SF"]
})
# Select columns by name
df.select("name", "age")
# Select columns with pl.col
df.select(pl.col("name"), pl.col("age"))
# Create a derived column while selecting
df.select(pl.col("name"), (pl.col("age") * 12).alias("age_months"))select is useful when you want to narrow a DataFrame to specific columns or compute derived columns without keeping the original columns.
Filtering Rows with filter
The filter method keeps rows that satisfy a boolean expression. To use multiple conditions, combine them with & (and) or | (or), and wrap each condition in parentheses because of operator precedence.
# Rows where age is greater than 30
df.filter(pl.col("age") > 30)
# Multiple conditions
df.filter((pl.col("age") > 25) & (pl.col("city") == "NYC"))filter returns a new DataFrame with only the matching rows. Because Polars DataFrames are immutable, the original DataFrame is unchanged.
Adding or Modifying Columns with with_columns
with_columns adds new columns or replaces existing ones. It accepts one or more expressions. If an expression uses alias, the result is named by that alias; otherwise, Polars uses the expression's name.
# Add a new column
df.with_columns((pl.col("age") * 2).alias("age_double"))
# Replace an existing column
df.with_columns((pl.col("age") + 1).alias("age"))
# Add multiple columns at once
df.with_columns([
(pl.col("age") * 2).alias("age_double"),
pl.col("city").str.to_uppercase().alias("city_upper")
])Use with_columns when you need to keep all existing columns and add derived ones.
Combining select, filter, and with_columns in a Pipeline
These methods return new DataFrames, so you can chain them. For example, filter rows, add a derived column, and then select a subset of columns:
result = (df
.filter(pl.col("age") > 25)
.with_columns((pl.col("age") * 12).alias("age_months"))
.select("name", "age_months"))This is a common pattern in data pipelines. In eager mode, each step executes immediately. In lazy mode, the whole pipeline is optimized before collection.
Performance Considerations: Lazy vs Eager Evaluation
Polars supports eager and lazy execution. In eager mode, operations like select, filter, and with_columns run immediately. In lazy mode, you build a query plan and call collect() at the end so Polars can optimize the entire pipeline, for example by pushing filters earlier and reducing unnecessary work.
# Lazy pipeline
lazy_result = (df.lazy()
.filter(pl.col("age") > 25)
.with_columns((pl.col("age") * 12).alias("age_months"))
.select("name", "age_months")
.collect())Lazy evaluation can improve performance on large datasets because Polars can reorder operations and skip unnecessary computations. On very small DataFrames, the overhead may not matter, but for production pipelines lazy evaluation is often preferred.
Common Mistakes and How to Avoid Them
A common mistake is using select when you need to keep existing columns. select returns only the columns you specify; if you want to add columns without dropping others, use with_columns.
Another common mistake is forgetting parentheses when combining conditions in filter. Because & and | have higher precedence than comparison operators, wrap each condition in parentheses.
# Wrong: missing parentheses
df.filter(pl.col("age") > 25 & pl.col("city") == "NYC") # raises an error
# Correct
df.filter((pl.col("age") > 25) & (pl.col("city") == "NYC"))Also, when using with_columns to replace a column, the expression must produce a column with the same name as the original. If you use alias with a different name, Polars adds a new column instead of replacing the old one.
Advanced Expression Patterns
Expressions can be nested and combined to create complex transformations. For example, pl.when and otherwise create conditional columns:
df.with_columns(
pl.when(pl.col("age") >= 30)
.then(pl.lit("senior"))
.otherwise(pl.lit("junior"))
.alias("level")
)You can also use expressions inside select, although aggregate calculations often need group_by. The main benefit is composability: expressions let you build complex logic without breaking the pipeline.
When working with many columns, pl.col("*") selects all columns, and a regex expression such as pl.col("^(name|age)$") selects columns matching the pattern. Regex selection is useful in select and with_columns when several columns share a naming convention and need the same transformation.
Try matching browser tools
All toolsSmall utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Python guides
All Python guidesPython iter(): How the Built-in Function Works
Explains Python's iter() built-in: how it creates iterators, the two-argument sentinel form, custom iterator classes, and common edge cases.
Python Pika: Queues, Exchanges, Routing Keys, and Acknowledgements
Learn how to use Python's Pika client with RabbitMQ: declare queues and exchanges, publish with routing keys, consume messages, and handle acknowledgements reliably.
Python Kafka: kafka-python vs Confluent Client
python kafka python vs confluent kafka: Compare kafka-python and Confluent Kafka Python clients: API differences, performance, feature coverage, and when to choose eac...
Python Kafka Offsets, Partitions, and Manual Commits
Learn how Kafka offsets and partitions work and how to manually commit offsets in Python consumers with kafka-python and confluent-kafka. Covers sync vs async commits, rebalances, and seeking.
More in Python
All Python guidesThe guides filed next to this one in Python, in library order.