Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Loguru Structured JSON Logging

Configure Loguru to emit structured JSON logs for machine parsing. This tutorial covers contextual fields, exception serialization, filtering, and production considerations.

~4 min readPython
#Loguru#JSON logging#Structured logging#Python logging
A developer configuring Loguru to output structured JSON log lines for machine parsing.

Loguru's default output is human-readable text with colors and formatting. For centralized logging systems like Elasticsearch, Loki, or CloudWatch, structured key-value pairs are easier to parse. JSON is a common interchange format, so you may want to configure Loguru to emit JSON lines. This article shows how to do that without external plugins.

Why Loguru Needs a JSON Formatter

Loguru's default sink prints a formatted string with time, level, message, and other fields. That works for local debugging but is less useful when logs are shipped to an aggregator. A JSON line is self-describing: each field has a name, and the structure is consistent across records. Aggregators can index fields without custom parsing rules. The tradeoff is that JSON is more verbose and slightly harder to read in a terminal, but that is acceptable when logs are consumed by machines.

A Minimal JSON Sink

Loguru lets you replace the default sink with a callable. The callable receives a Message object; message.record is the dictionary containing Loguru's metadata. Inside the function, build a dictionary and serialize it with json.dumps. Here is a minimal implementation:

python
import json import sys from loguru import logger def json_sink(message): record = message.record log_entry = { "time": record["time"].isoformat(), "level": record["level"].name, "message": record["message"], "module": record["module"], "function": record["function"], "line": record["line"], } print(json.dumps(log_entry), file=sys.stderr) logger.remove() logger.add(json_sink)

The logger.remove() call removes the default stderr sink. The custom sink receives a Message object whose record dictionary contains all the metadata Loguru collects. The time value is a datetime object, so call isoformat() to get a string. level is a Level object; .name gives the uppercase level name. message is the formatted message after interpolation.

This sink writes to stderr to match Loguru's default behavior. If you want stdout, change the file argument to sys.stdout. Each log call produces one JSON object on its own line.

Capturing Contextual Fields with bind and extra

Loguru's bind method returns a logger with extra fields attached. These fields end up in record["extra"]. To include them in JSON output, merge the extra dictionary into the log entry:

python
import json import sys from loguru import logger def json_sink(message): record = message.record log_entry = { "time": record["time"].isoformat(), "level": record["level"].name, "message": record["message"], **record["extra"], } print(json.dumps(log_entry), file=sys.stderr) logger.remove() logger.add(json_sink) logger.bind(user_id=123, request_id="abc").info("user logged in")

The output is a single JSON line with user_id and request_id alongside the standard fields. **record["extra"] works because extra is a regular dictionary. If a contextual key collides with a standard field, placing the unpacking after the standard keys lets the contextual value take precedence. This is useful for request IDs, user IDs, or domain-specific context without changing the sink.

Serializing Exceptions and Tracebacks

When you call logger.exception or logger.opt(exception=True), Loguru captures the active exception and stores it in record["exception"]. This field is a tuple containing the exception type, value, and traceback object. To serialize it, convert it to a JSON-friendly structure:

python
import json import sys import traceback from loguru import logger def json_sink(message): record = message.record log_entry = { "time": record["time"].isoformat(), "level": record["level"].name, "message": record["message"], **record["extra"], } if record["exception"] is not None: exc_type, exc_value, exc_tb = record["exception"] log_entry["exception"] = { "type": exc_type.__name__, "value": str(exc_value), "traceback": "".join(traceback.format_tb(exc_tb)).strip(), } print(json.dumps(log_entry), file=sys.stderr) logger.remove() logger.add(json_sink) try: 1 / 0 except ZeroDivisionError: logger.exception("division failed")

Here exc_type is the exception class, exc_value is the instance, and exc_tb is a traceback object. traceback.format_tb(exc_tb) returns a list of strings, which we join into a single string. strip() removes trailing newlines. This structure is easy to query in an aggregator: filter by exception.type or search for a substring in exception.traceback.

Filtering and Multiple Sinks

You often want JSON output for production and human-readable output for local development. Loguru allows multiple sinks with different filters. The filter parameter can be a callable that receives a record and returns True if the record should be processed by that sink. For example:

python
import json import sys from loguru import logger def json_sink(message): record = message.record log_entry = { "time": record["time"].isoformat(), "level": record["level"].name, "message": record["message"], **record["extra"], } print(json.dumps(log_entry), file=sys.stderr) logger.remove() logger.add(sys.stderr, format="{time} {level} {message}", filter=lambda r: r["level"].name == "DEBUG") logger.add(json_sink, filter=lambda r: r["level"].name != "DEBUG")

In this setup, DEBUG messages go to the console in a readable format, while INFO and above go to the JSON sink. The filter receives the full record, so you can filter on module name, thread ID, or any field in record["extra"]. This is useful for separating noisy debug logs from operational logs.

Performance and Serialization Overhead

JSON serialization adds CPU cost per log call. For moderate log volumes, json.dumps is sufficient. For high throughput, consider using orjson if you can add a dependency. orjson is often faster and natively serializes datetime objects. However, a callable sink is invoked synchronously, so a slow serializer blocks the calling thread. If you need non-blocking logging, logger.add(..., enqueue=True) moves the write to a background thread; a manual queue adds complexity.

Another concern is the serializability of values in record["extra"]. If you bind a custom object, json.dumps will fail. Use default=str in json.dumps to fall back to string conversion, or ensure bound values are primitives. For example:

python
print(json.dumps(log_entry, default=str), file=sys.stderr)

This prevents a TypeError when an unexpected object appears. The cost is that the object's string representation may not be what you want, so it is better to explicitly convert non-serializable values before binding them.

Operational Considerations for JSON Logs

Most log shippers expect one JSON object per line. The examples in this article use print(json.dumps(...)), which emits one physical line per record. json.dumps escapes newline characters inside strings, so a message containing a newline remains valid single-line JSON. Do not manually replace newlines in values before passing them to json.dumps; that changes the decoded value to a literal \n (backslash followed by n) instead of a newline.

Log rotation applies to file sinks, not to a custom callable sink that writes to a stream. If you want JSON files with rotation and retention, pass a file path to logger.add and provide a format function that returns a JSON string:

python
def json_format(record): log_entry = { "time": record["time"].isoformat(), "level": record["level"].name, "message": record["message"], **record["extra"], } return json.dumps(log_entry) + "\n" logger.add("app.json", format=json_format, rotation="10 MB", retention="30 days")

Here Loguru manages the file lifecycle, so the rotation and retention settings apply. For brevity, this example does not include exception serialization; extend json_format with the same logic shown earlier if you need it.

Finally, ensure the sink writes to the correct stream for your deployment. In containers, stdout is often the standard for logs. Using sys.stdout makes logs appear in docker logs or kubectl logs. If your infrastructure reads stderr instead, adjust accordingly. The choice affects how your orchestrator collects logs, so match the platform convention.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.