Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Redis Pub/Sub: Implementation and Patterns

Learn how to implement Redis pub/sub in Python with redis-py: publishing, subscribing, pattern subscriptions, error handling, reconnection, and operational tradeoffs.

~4 min readPython
#Redis#Pub/Sub#Python#Messaging
Diagram of Python code publishing and subscribing to Redis channels for real-time messaging.

Redis pub/sub is a lightweight messaging pattern in which publishers send messages to channels and subscribers receive messages without the publisher knowing who is listening. In Python, redis-py exposes this through a PubSub object. This article covers the core API and practical patterns for publishing, subscribing, pattern matching, error handling, and operational considerations.

Setting Up a Redis Connection for Pub/Sub

The redis-py library provides a straightforward API for Redis pub/sub. Start by creating a Redis client instance and then call pubsub() to get a PubSub object.

python
import redis client = redis.Redis(host='localhost', port=6379, decode_responses=True) pubsub = client.pubsub()

The decode_responses=True parameter makes Redis return strings instead of bytes, which is usually more convenient for JSON payloads or plain text. If you omit it, every message arrives as bytes and you must decode manually.

The PubSub object is separate from the main client. It maintains its own connection to Redis and manages the subscription state. You can have multiple PubSub instances on the same client, but each active PubSub uses its own connection.

Publishing Messages

Publishing is done through the main client, not the PubSub object. The publish method takes a channel name and a message:

python
client.publish('notifications', 'user-42 signed up')

The message can be any string. If you need to send structured data, serialize it to JSON first:

python
import json payload = {'user_id': 42, 'action': 'signup'} client.publish('events', json.dumps(payload))

publish returns the number of subscribers that received the message. A return value of 0 means no subscriber was listening on that channel at that moment. This is useful for debugging, but remember that Redis pub/sub is fire-and-forget: if no subscriber is connected, the message is simply dropped.

Subscribing to Channels

To receive messages, subscribe to a channel and then read from the PubSub object. The simplest way is to use get_message() in a loop:

python
pubsub.subscribe('notifications') while True: message = pubsub.get_message() if message and message['type'] == 'message': print(message['data'])

get_message() returns a dictionary with keys like type, channel, and data. It is non-blocking: if no message is available, it returns None (or a subscribe confirmation message on the first call). For a blocking loop, use listen() instead:

python
for message in pubsub.listen(): if message['type'] == 'message': print(message['data'])

listen() yields messages as they arrive and blocks the current thread. This is the pattern most applications use when they need a dedicated subscriber thread. For exact channel subscriptions, delivered messages have type equal to 'message'; pattern subscriptions deliver 'pmessage' entries, as described later.

Handling Message Delivery and Threading

Because listen() blocks, you typically run it in a separate thread or process. The PubSub object is not thread-safe, so you should not call subscribe() from one thread while another thread is iterating over listen(). A common design is to create the subscription in the same thread that consumes messages.

If you need to stop a subscriber, you can call pubsub.close() from another thread. This unblocks listen() and raises an exception inside the loop, so you must handle it:

python
import threading def consume(): pubsub.subscribe('notifications') try: for message in pubsub.listen(): if message['type'] == 'message': handle(message['data']) except redis.exceptions.ConnectionError: pass thread = threading.Thread(target=consume) thread.start()

Closing the pubsub connection from the main thread causes listen() to raise a ConnectionError, which you can catch to exit cleanly. Because PubSub is not thread-safe, use this technique only as a shutdown signal and avoid other concurrent calls on the same object.

Pattern Subscriptions

Redis also supports pattern-based subscriptions with psubscribe(). This lets you subscribe to multiple channels that match a glob-style pattern:

python
pubsub.psubscribe('events:*')

Now every message published to events:user-42 or events:order-7 will be delivered. Pattern messages arrive with type set to 'pmessage'; the message['channel'] field contains the actual channel name, and message['pattern'] contains the pattern that matched:

python
for message in pubsub.listen(): if message['type'] == 'pmessage': print(message['channel'], message['data'])

This is useful for routing messages by topic without maintaining an explicit list of channels.

Pattern subscriptions have a higher overhead than exact subscriptions because Redis has to evaluate the pattern for every publish. Use them when the set of channels is dynamic or too large to enumerate.

Error Handling and Reconnection

Redis pub/sub connections can fail for many reasons: network timeouts, Redis restarts, or dropped connections. The redis-py client does not automatically reconnect a PubSub object. If the connection drops, listen() raises a ConnectionError.

A robust subscriber should catch the exception and recreate the PubSub object:

python
import time while True: try: pubsub = client.pubsub() pubsub.subscribe('notifications') for message in pubsub.listen(): if message['type'] == 'message': handle(message['data']) except redis.exceptions.ConnectionError: time.sleep(1)

This loop reconnects after a failure, but it does not replay messages that were missed during the outage. If you are using close() to shut down intentionally, coordinate with a flag or break the loop so it does not reconnect. Redis pub/sub has no persistence or acknowledgment mechanism, so any message published while the subscriber is disconnected is lost. If you need reliable delivery, use a different pattern, such as Redis Streams or a message queue.

Performance and Operational Considerations

Pub/sub is designed for low-latency fan-out, not for durable message processing. The main operational tradeoff is that messages are not stored. If your subscriber goes offline, it misses everything published during that period.

The number of subscribers on a channel affects throughput. Redis delivers each message to every subscriber over a separate connection, so a channel with 100 subscribers requires 100 writes. For high fan-out, consider whether Redis is the right fit or whether a broker designed for large fan-out is more appropriate.

On the Python side, the GIL can limit throughput if you process messages in a single thread. For CPU-bound processing, offload the work to a worker pool and let the subscriber thread only dispatch messages.

Finally, monitor the Redis server's memory and connection count. An active PubSub subscription holds a connection open, and a large number of subscribers can exhaust the connection limit. Use connection pooling carefully and close PubSub objects when they are no longer needed.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.