Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Loguru File Logging: Rotation and Retention

Configure Loguru file logging with rotation and retention: size and time rotation options, retention policies, compression, and common production pitfalls.

~6 min readPython
#Loguru#Python Logging#Log Rotation#Log Retention
Illustration of rotating log files with a retention window, showing old files being archived and deleted over time.

Loguru's rotation and retention options keep file-based logs from growing without bound. rotation decides when to close the current log file and start a new one; retention decides which older files to delete. This article explains how to configure both, what the options do at runtime, and where the behavior can surprise you in production.

Adding a File Sink with Rotation and Retention

Loguru writes to files through logger.add(), which returns a handler ID. The simplest file sink looks like this:

python
from loguru import logger logger.add("app.log")

Without rotation, app.log grows indefinitely. Pass a rotation argument to bound the file size:

python
logger.add("app.log", rotation="500 MB")

When the current file reaches 500 MB, Loguru closes it and starts a new one. The old file is renamed with a timestamp, for example app.log.2025-03-14_12-30-00_000000. The retention argument then controls how many of those older files are kept:

python
logger.add("app.log", rotation="500 MB", retention="10 days")

This combination is the core pattern: size-based rotation keeps each file manageable, and time-based retention prevents an unbounded collection of archived files.

How Rotation Works

rotation accepts several forms:

  • A string with a size, like "100 MB" or "1 GB".
  • A string with a time interval, like "1 day", "12 hours", or "1 week".
  • A specific time of day, like "12:00".
  • A callable that returns True when the file should rotate.

With a size, Loguru checks the file size while messages are written and rotates once the threshold is crossed. With a time interval, the check is based on the current time and the time the file was created, and it only happens while a message is being logged. A process that logs nothing for a long time does not rotate its file on schedule. For "12:00", Loguru rotates at the next noon it observes while processing a message.

A callable gives you full control. It is called before each message is logged with the message dict and the current file object, and returns True to rotate:

python
def rotate_at_noon(message, file): return message["record"]["time"].hour == 12 logger.add("app.log", rotation=rotate_at_noon)

Use a callable when the built-in string options are not enough, such as rotating only for certain log levels or message content.

How Retention Works

retention removes old log files. It accepts:

  • A timedelta object, like timedelta(days=7).
  • A string with a time interval, like "7 days" or "1 month".
  • An integer, which is the number of files to keep.
  • A callable that receives a list of file paths and returns the ones to delete.

When retention is an integer, Loguru keeps that many recent files. For example, retention=5 keeps the current file and the four most recent rotated files each time it applies the policy. This is often more predictable than time-based retention when log volume varies.

Time-based retention deletes files whose modification time is older than the interval. The check happens when a sink is added and when a rotation occurs; it is not a background job. Files that become stale while the process runs are cleaned up at the next rotation.

A callable gives you complete control over which files to delete:

python
def clean_old_logs(file_paths): return [p for p in file_paths if "debug" in p] logger.add("app.log", retention=clean_old_logs)

This is useful when you want to keep certain files longer or apply a different policy than the built-in options.

Combining Rotation and Retention

Rotation and retention work independently. Rotation decides when to create a new file; retention decides which old files to delete. They are often used together, but you can use only one.

A common pattern is size-based rotation with a retention count:

python
logger.add("app.log", rotation="50 MB", retention=20)

This keeps the log directory bounded to roughly 1 GB (20 files × 50 MB). Time-based retention with time-based rotation can be simpler to reason about when your log volume is steady:

python
logger.add("app.log", rotation="1 day", retention="30 days")

This keeps about a month of daily files. The exact number of files depends on whether the process runs continuously and when messages are logged.

One subtlety: retention is based on the file name pattern derived from the path you pass to add(), not on a record of which process created each file. If you add files manually and they match that pattern, Loguru may include them in retention decisions.

Compression and Other File Options

Loguru can compress rotated files automatically with the compression parameter. Supported formats include gz, bz2, xz, and zip. For example:

python
logger.add("app.log", rotation="100 MB", retention="30 days", compression="gz")

When a file is rotated, Loguru compresses it and adds the extension .gz. The original uncompressed file is removed. This reduces disk usage but adds CPU overhead during rotation. For high-volume logs, consider whether compression is worth the cost.

Other useful file options include enqueue=True, which sends log messages through a separate thread and makes writes thread-safe, and backtrace and diagnose for exception details. The enqueue option is especially relevant when multiple threads write to the same sink.

Common Pitfalls with Rotation and Retention

Rotation Does Not Trigger on Process Exit

Loguru does not force a rotation on shutdown. If the process exits before the next rotation check, the current file remains as-is. Removing the handler with logger.remove() closes the file, but it does not rename it with a rotation timestamp. If you need an archived file on exit, close the sink and rename the file yourself.

Retention Runs at Startup and Rotation

Retention is not a background job. It runs when a sink is added and when a rotation occurs. If your process runs for a long time without rotating, old files are not cleaned up. If you rely on time-based retention, make sure rotation happens frequently enough to trigger cleanup.

Time Zone and DST

Time-based rotation uses local time by default. If your server changes time zones or observes daylight saving time, a "12:00" rotation may occur at an unexpected moment. For most applications this is acceptable, but if you need strict UTC-based rotation, use a callable that checks the current UTC time.

Retention with a Callable Can Delete the Current File

A retention callable receives files that match the sink pattern, which can include the file currently being written. If it returns that file, Loguru may delete it, which can cause errors on subsequent writes. Filter the active file out of the list unless you intentionally want it removed.

Changing the Pattern Breaks Retention

If you change the file name pattern in add(), Loguru derives a new retention pattern from the new path. Old files from the previous pattern are not covered by the new sink's retention. Clean up files from an old pattern manually or with a separate routine.

Production Considerations

In production, the most important decision is whether to use size-based or time-based rotation. Size-based rotation keeps each file under a predictable maximum, which is useful when logs are shipped to an external system with a file size limit. Time-based rotation makes it easier to correlate logs with a specific date, which is useful for debugging and auditing.

Retention should match your compliance or debugging needs. If you are required to keep logs for a certain period, use time-based retention with a margin. If you only need the most recent logs, a retention count is simpler and prevents disk usage from growing with log volume.

Also consider the interaction between rotation and the rest of your logging pipeline. If you use enqueue=True, messages are written asynchronously, so rotation may happen slightly after the size or time threshold is observed. This is usually fine, but if you need exact file boundaries, use a synchronous sink.

Finally, monitor the log directory. Even with rotation and retention, a misconfigured sink can fill the disk. A simple scheduled check or monitoring alert on the total size of the log directory can catch problems before they affect the application.

Loguru's rotation and retention give you a compact way to manage file growth without external tools. Understanding how the options behave at runtime lets you choose the right combination for your workload and avoid the pitfalls that appear only after a process has been running for a while.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.