Python BeautifulSoup: find, find_all, and CSS Selectors
Learn when to use find, find_all, and CSS selectors in BeautifulSoup. Compare syntax, performance, and practical decision criteria.

When parsing HTML with Python's BeautifulSoup library, you can locate elements with the find/find_all methods or with CSS selectors through select/select_one. Choosing between the two depends on the structure of your query, the kind of filtering you need, and how readable you want the scraping code to be.
How find() and find_all() Work
The find() method returns the first matching tag, while find_all() returns a ResultSet (a list subclass) of all matching tags. Both accept a tag name, attribute filters, text filters, or a combination of filters. For example:
from bs4 import BeautifulSoup
html = """
<div class="product">
<span class="price">$19.99</span>
<span class="name">Widget</span>
</div>
<div class="product">
<span class="price">$29.99</span>
<span class="name">Gadget</span>
</div>
"""
soup = BeautifulSoup(html, 'html.parser')
first_price = soup.find('span', class_='price')
all_prices = soup.find_all('span', class_='price')find() returns a Tag object or None if no match exists. find_all() always returns a list-like object, even when empty. This distinction matters when you chain methods or check for existence.
CSS Selectors with select() and select_one()
BeautifulSoup's select() method accepts a CSS selector string and returns a list of matching elements. select_one() returns the first match or None. CSS selectors are often more expressive for complex structural queries:
# Equivalent to the previous example
first_price = soup.select_one('span.price')
all_prices = soup.select('span.price')You can use descendant combinators, attribute selectors, and pseudo-classes such as :nth-of-type(2). This often reduces code verbosity compared to nested find_all calls.
Comparing find/find_all and CSS Selectors
| Feature | find/find_all | select/select_one |
|---|---|---|
| Syntax | Method calls with filters | CSS selector string |
| Return type | Tag or list of Tags | Tag or list of Tags |
| Expressiveness | Limited to method filters | Complex CSS selector syntax |
| Readability | Verbose for nested queries | Compact for complex paths |
| Performance | Traverses with Python filters | Parses CSS, then traverses |
| Best use case | Simple lookups, dynamic filters | Complex structural queries |
Both approaches traverse the parsed tree. find_all applies the Python filter you provide, while select first parses the CSS selector string. For ordinary scraping scripts the difference is usually negligible. If performance matters, measure and optimize only after confirming a bottleneck.
When to Use CSS Selectors Over find_all
Use CSS selectors when your query involves hierarchy, attribute conditions, or pseudo-classes. For example, extracting all links inside a specific section:
# With find_all
links = []
for section in soup.find_all('div', class_='content'):
links.extend(section.find_all('a'))
# With select
links = soup.select('div.content a')The select version is shorter and directly expresses the intended structure. If you need to filter by attribute value, CSS selectors are often clearer:
# find_all with attribute filter
inputs = soup.find_all('input', attrs={'type': 'text'})
# select with attribute filter
inputs = soup.select('input[type="text"]')Performance and Parsing Overhead
Both approaches traverse the parsed tree once a query is built. select also has to process the selector string, so repeated use of the same selector in a hot loop can be worth optimizing. SoupSieve, the selector engine used by BeautifulSoup, lets you compile a selector once and reuse it:
import soupsieve
compiled = soupsieve.compile('div.product > span.price')
# Later, reuse compiled on any soup object
prices = compiled.select(soup)This avoids repeated work in code that runs the same query many times. For typical scraping scripts, the difference is rarely noticeable; do not optimize unless you have profiled the code.
Common Pitfalls with Class Names and Nested Elements
A common mistake is using class instead of class_ in find_all. Python reserves class, so BeautifulSoup uses class_ as the argument name. In CSS selectors, dot notation is natural for classes.
Another pitfall is handling multiple class names. CSS selectors express class-list conditions directly:
# Element has both 'a' and 'b' classes
soup.select('.a.b')
# Element has either 'a' or 'b' class
soup.select('.a, .b')With find_all, these conditions require a custom predicate or more verbose filtering, so CSS is usually the better choice for class-list logic. Nested element queries also differ: find_all often needs explicit loops or list comprehensions, while select uses descendant combinators directly.
Practical Example: Scraping a Table with Both Approaches
Consider an HTML table with rows and cells. You want to extract all cell text from the second column.
html = """
<table>
<tr><td>Name</td><td>Price</td></tr>
<tr><td>Widget</td><td>$10</td></tr>
<tr><td>Gadget</td><td>$20</td></tr>
</table>
"""
soup = BeautifulSoup(html, 'html.parser')
# Using find_all
prices = []
for row in soup.find_all('tr'):
cells = row.find_all('td')
if len(cells) > 1:
prices.append(cells[1].text)
# Using select
prices = [td.text for td in soup.select('tr td:nth-of-type(2)')]The select version is more concise and directly targets the second cell without manual index checks. It also avoids issues with rows that have fewer cells.
Choosing the Right Method for Your Scraper
There is no universal winner. Use find_all when you need to filter by dynamic Python conditions that are hard to express in CSS, such as a custom predicate over attribute values. Use select when your query is structural and static, especially if you are comfortable with CSS. For complex paths, select often reads like the document structure. For simple lookups, either works; pick the one that makes the code clearer to your team.
If you are already using BeautifulSoup, both methods operate on the same Tag objects, so you can mix them freely. The key is to choose the approach that minimizes cognitive overhead for the specific query you are writing.
Try matching browser tools
All toolsSmall utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Python guides
All Python guidesPython iter(): How the Built-in Function Works
Explains Python's iter() built-in: how it creates iterators, the two-argument sentinel form, custom iterator classes, and common edge cases.
Python Pika: Queues, Exchanges, Routing Keys, and Acknowledgements
Learn how to use Python's Pika client with RabbitMQ: declare queues and exchanges, publish with routing keys, consume messages, and handle acknowledgements reliably.
Python Kafka: kafka-python vs Confluent Client
python kafka python vs confluent kafka: Compare kafka-python and Confluent Kafka Python clients: API differences, performance, feature coverage, and when to choose eac...
Python Kafka Offsets, Partitions, and Manual Commits
Learn how Kafka offsets and partitions work and how to manually commit offsets in Python consumers with kafka-python and confluent-kafka. Covers sync vs async commits, rebalances, and seeking.
More in Python
All Python guidesThe guides filed next to this one in Python, in library order.