Python Boto3 CloudWatch: Sending Custom Metrics
Learn to publish custom CloudWatch metrics with Python Boto3: send metric data, create alarms, retrieve statistics, and avoid common production pitfalls.

With CloudWatch custom metrics, you can monitor application-specific numbers that AWS does not track by default. The Python Boto3 library provides a straightforward interface to CloudWatch for this purpose. This article covers the core operations: sending metric data, creating alarms, and querying statistics, with code examples and operational details that matter in production.
Setting Up the CloudWatch Client
The first step is to create a Boto3 client for CloudWatch. The client requires valid AWS credentials, which can come from environment variables, IAM roles, or the shared credentials file. The region must match where your metrics and alarms should reside.
import boto3
cloudwatch = boto3.client('cloudwatch', region_name='us-east-1')If you're running on EC2 or Lambda, the default credential chain picks up the instance role or execution role automatically. For local development, you might need to set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY explicitly. The client is thread-safe, so you can reuse it across concurrent calls.
Publishing Custom Metrics with put_metric_data
The primary method for sending metrics is put_metric_data. It accepts a namespace at the top level and one or more metric data entries, each containing a metric name, value, timestamp, and optional dimensions and unit. Namespaces isolate metrics from different applications; choose a meaningful one like MyApp/Transactions.
from datetime import datetime, timezone
response = cloudwatch.put_metric_data(
Namespace='MyApp/Transactions',
MetricData=[
{
'MetricName': 'ProcessingTime',
'Value': 125.4,
'Unit': 'Milliseconds',
'Timestamp': datetime.now(timezone.utc),
'Dimensions': [
{'Name': 'Environment', 'Value': 'production'},
{'Name': 'Service', 'Value': 'checkout'}
]
}
]
)Dimensions are key-value pairs that let you filter and aggregate metrics later. For example, you can query the same metric across different services. The Timestamp is optional; if omitted, CloudWatch uses the time the request was received. For high-frequency metrics, you can send up to 20 values in a single request by adding more entries to MetricData.
Handling Metric Data Limits and Retries
Each put_metric_data call can contain up to 20 metric data entries. If you need to send more, split the data across multiple calls. CloudWatch also accepts a StorageResolution field on each metric data entry to specify whether the metric is standard (60-second) or high-resolution (1-second). High-resolution metrics cost more, so choose carefully.
Boto3 automatically retries on transient failures like throttling or network errors. The default retry mode is legacy, which retries up to 5 times with exponential backoff. For production workloads, consider configuring a more aggressive retry strategy via the Config object:
from botocore.config import Config
config = Config(
retries={'max_attempts': 10, 'mode': 'standard'}
)
cloudwatch = boto3.client('cloudwatch', config=config)The standard retry mode also handles throttling by inspecting the ThrottlingException response and waiting accordingly. Retries can cause duplicate metric data if the first request succeeded but the response was lost. Design your metric ingestion to tolerate duplicates where duplicates are not meaningful.
Creating CloudWatch Alarms with put_metric_alarm
Once you're publishing metrics, you often want to trigger actions when a threshold is breached. put_metric_alarm creates or updates an alarm based on a metric. You specify the metric name, namespace, dimensions, statistic, period, threshold, and comparison operator.
cloudwatch.put_metric_alarm(
AlarmName='CheckoutProcessingTimeHigh',
AlarmDescription='Alert when checkout processing time exceeds 500ms',
MetricName='ProcessingTime',
Namespace='MyApp/Transactions',
Statistic='Average',
Dimensions=[
{'Name': 'Environment', 'Value': 'production'},
{'Name': 'Service', 'Value': 'checkout'}
],
Period=60,
EvaluationPeriods=2,
Threshold=500.0,
ComparisonOperator='GreaterThanThreshold',
AlarmActions=['arn:aws:sns:us-east-1:123456789012:my-topic']
)Period is the length of time in seconds for each data point. EvaluationPeriods defines how many consecutive periods must breach the threshold before the alarm fires. The AlarmActions list can contain SNS topic ARNs, Auto Scaling policies, or EC2 actions. You can also set OKActions and InsufficientDataActions to notify on recovery or missing data.
When you update an existing alarm, the request is treated as the full desired state. If you need to change only one parameter, resend the other required fields as well; omitted parameters are reset to defaults. Alarm names must be unique within an AWS account and region.
Retrieving Metrics with get_metric_statistics
To inspect historical data, use get_metric_statistics. This method returns a list of data points for a given metric over a specified time range. You must provide the namespace, metric name, dimensions, start time, end time, period, and statistics.
from datetime import datetime, timedelta, timezone
response = cloudwatch.get_metric_statistics(
Namespace='MyApp/Transactions',
MetricName='ProcessingTime',
Dimensions=[
{'Name': 'Environment', 'Value': 'production'},
{'Name': 'Service', 'Value': 'checkout'}
],
StartTime=datetime.now(timezone.utc) - timedelta(hours=1),
EndTime=datetime.now(timezone.utc),
Period=300,
Statistics=['Average', 'Maximum', 'SampleCount']
)
datapoints = response['Datapoints']The response contains a list of data points, each with a timestamp and the requested statistics. Note that CloudWatch aggregates raw data into the specified period. The period must be at least the metric's storage resolution: 60 seconds for standard metrics, or 1 second for high-resolution metrics. Requesting a shorter period can return empty or incomplete results.
Listing Metrics and Handling Pagination
The list_metrics operation returns all metrics that match a given namespace, metric name, or dimensions. This is useful for discovering what metrics exist or for building dashboards dynamically.
paginator = cloudwatch.get_paginator('list_metrics')
for page in paginator.paginate(Namespace='MyApp/Transactions'):
for metric in page['Metrics']:
print(metric['MetricName'], metric.get('Dimensions'))The response is paginated; using the paginator handles multiple pages automatically. Each metric object includes the namespace, metric name, and dimensions.
Error Handling and Boto3 Retry Configuration
Boto3 raises exceptions from the botocore.exceptions module. Common ones for CloudWatch include InvalidParameterValue, MissingParameter, LimitExceeded, and Throttling. Always catch these and handle them appropriately, especially in production scripts.
from botocore.exceptions import ClientError
try:
cloudwatch.put_metric_data(...)
except ClientError as e:
if e.response['Error']['Code'] == 'Throttling':
# Apply backoff or queue for later
pass
else:
raiseThrottling occurs when you exceed the API request rate. The account-level default for put_metric_data is 150 requests per second, but this quota can be increased. If you're sending many small batches, consider aggregating values into fewer requests to stay within the limit. Also, ensure your IAM role has the required permissions: cloudwatch:PutMetricData, cloudwatch:PutMetricAlarm, and cloudwatch:GetMetricStatistics. Using overly broad permissions like cloudwatch:* is convenient but not recommended for production.
Operational Considerations for Production
Custom metrics can incur charges, and high-resolution metrics cost more. Before publishing a high-cardinality metric, think about whether you really need per-instance or per-user dimensions. A metric with many unique dimension combinations can become expensive. Instead, aggregate data client-side or use a single metric with a Service dimension for a few known values.
Also consider retention. CloudWatch stores all metrics for 15 months, so you don't need to archive them yourself. However, if you need to query metrics frequently, using get_metric_statistics with a period of 60 seconds over a long range can be slow. For dashboards, use CloudWatch's built-in metric math instead of pulling raw data into your application.
When publishing metrics from a high-throughput service, batch values into a single put_metric_data call every few seconds rather than sending one request per event. This reduces API calls and throttling risk. Use the Timestamp field to preserve the actual event time; otherwise all points will appear at the request time and the metrics will look flat.
Finally, test your alarm thresholds with real data before deploying. A common mistake is setting EvaluationPeriods too high, which delays alerts, or too low, which causes flapping. Use CloudWatch's alarm history to verify that the alarm transitions correctly from INSUFFICIENT_DATA to OK to ALARM as you simulate load.
Try matching browser tools
All toolsSmall utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Python guides
All Python guidesPython iter(): How the Built-in Function Works
Explains Python's iter() built-in: how it creates iterators, the two-argument sentinel form, custom iterator classes, and common edge cases.
Python Pika: Queues, Exchanges, Routing Keys, and Acknowledgements
Learn how to use Python's Pika client with RabbitMQ: declare queues and exchanges, publish with routing keys, consume messages, and handle acknowledgements reliably.
Python Kafka: kafka-python vs Confluent Client
python kafka python vs confluent kafka: Compare kafka-python and Confluent Kafka Python clients: API differences, performance, feature coverage, and when to choose eac...
Python Kafka Offsets, Partitions, and Manual Commits
Learn how Kafka offsets and partitions work and how to manually commit offsets in Python consumers with kafka-python and confluent-kafka. Covers sync vs async commits, rebalances, and seeking.
More in Python
All Python guidesThe guides filed next to this one in Python, in library order.