Using Python Boto3 with AWS Secrets Manager
Use Python and boto3 to retrieve, create, and update secrets in AWS Secrets Manager, with error handling and caching patterns.

When you need to retrieve a secret from AWS Secrets Manager in Python, the boto3 SDK exposes the get_secret_value method on the Secrets Manager client. The standard flow is to create a client, call get_secret_value with the secret's name or ARN, and read the returned SecretString or SecretBinary field. This article covers the full lifecycle of working with secrets: retrieving values, parsing JSON payloads, creating and updating secrets, handling API errors, and caching values to avoid unnecessary calls.
Setting Up the boto3 Secrets Manager Client
The first step is to create a Secrets Manager client. boto3 resolves credentials from the standard chain: environment variables, the shared credentials file, or an IAM role attached to the compute resource.
import boto3
client = boto3.client("secretsmanager", region_name="us-east-1")The region_name matters because secrets are regional. A secret stored in us-east-1 is not accessible from a client pointing at eu-west-1. If your application runs on EC2, ECS, or Lambda, prefer an IAM role over hard-coded credentials so that permissions are managed centrally and rotated automatically.
Retrieving a Secret Value with get_secret_value
The core operation is get_secret_value. Pass the secret's name or ARN as SecretId.
response = client.get_secret_value(SecretId="my-database-password")
secret_string = response["SecretString"]Secrets Manager stores either a string or binary value. For string secrets, the response contains SecretString. For binary secrets, the response contains SecretBinary, which is a bytes object in Python.
response = client.get_secret_value(SecretId="my-tls-key")
binary_secret = response["SecretBinary"]By default, get_secret_value returns the version staged as AWSCURRENT. If you need a previous version, pass the VersionStage or VersionId parameter. In most application code, the default is what you want because it always reflects the latest rotated value.
Working with JSON Secrets
Storing multiple fields in a single secret is common. For example, a database credential may contain a username, password, and host. Store the value as a JSON string and parse it after retrieval.
import json
response = client.get_secret_value(SecretId="my-db-credentials")
secret = json.loads(response["SecretString"])
username = secret["username"]
password = secret["password"]
host = secret["host"]The json.loads call can raise JSONDecodeError if the secret is not valid JSON. If you control the secret's format, validate it at write time. If the secret may be plaintext, check whether the value parses as JSON before assuming a structure.
Creating and Updating Secrets Programmatically
Applications that provision their own infrastructure often create secrets at deployment time. create_secret stores a new secret and fails if the name already exists.
client.create_secret(
Name="my-api-credentials",
SecretString=json.dumps({"username": "admin", "password": "initial-password"})
)To change an existing secret, use update_secret. This creates a new version and stages it as AWSCURRENT.
client.update_secret(
SecretId="my-api-credentials",
SecretString=json.dumps({"username": "admin", "password": "rotated-password"})
)A common mistake is calling create_secret on every application start. If the secret already exists, the call raises ResourceExistsException. Decide whether your deployment should create or update, and branch accordingly.
Handling Common Errors
boto3 raises ClientError for API failures. The error code is available in the exception's response dictionary under Error.Code; branch on that code rather than catching the exception generically.
from botocore.exceptions import ClientError
try:
response = client.get_secret_value(SecretId="my-secret")
except ClientError as e:
code = e.response["Error"]["Code"]
if code == "ResourceNotFoundException":
print("Secret does not exist")
elif code == "AccessDeniedException":
print("Missing IAM permission for secretsmanager:GetSecretValue")
elif code == "InvalidRequestException":
print("Secret is scheduled for deletion or the request is malformed")
else:
raiseThe AccessDeniedException case deserves attention. It usually means the IAM policy attached to the caller does not grant secretsmanager:GetSecretValue on that resource. The else branch re-raises the exception so unexpected failures are not silently swallowed.
Caching Secrets to Reduce API Calls
Every get_secret_value call is an API request with latency and cost. In high-traffic services, fetching the secret on every request adds unnecessary overhead. A simple in-memory cache with a time-to-live avoids repeated calls while still picking up rotated values within the TTL window.
import time
class SecretCache:
def __init__(self, client, ttl_seconds=300):
self.client = client
self.ttl_seconds = ttl_seconds
self._cache = {}
def get(self, secret_id):
now = time.time()
if secret_id in self._cache:
value, expires_at = self._cache[secret_id]
if now < expires_at:
return value
response = self.client.get_secret_value(SecretId=secret_id)
value = response.get("SecretString")
if value is None:
value = response["SecretBinary"]
self._cache[secret_id] = (value, now + self.ttl_seconds)
return valueThe cache stores the secret value and an expiry timestamp. On the first call, the value is fetched and cached. The example handles both string and binary values: if SecretString is absent, it reads SecretBinary. Subsequent calls within the TTL return the cached value without hitting the API. After expiry, the next call refreshes the value. A TTL of 300 seconds balances freshness against API cost for most applications. If your secrets rotate frequently, lower the TTL; if they change rarely, raise it.
IAM Permissions and Security Considerations
The boto3 client only works if the caller has the right IAM permissions. A typical read policy grants secretsmanager:GetSecretValue on the specific secret ARN.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:my-secret-*"
}
]
}Secrets Manager protects secret values with KMS encryption. If the secret uses a customer-managed KMS key, the caller also needs kms:Decrypt permission on that key.
Writing secrets requires additional actions such as secretsmanager:CreateSecret and secretsmanager:UpdateSecret. Scope the resource to the secrets your application actually manages rather than using a wildcard across the account.
Never log the value of a secret. Log the secret's name, the fact that retrieval succeeded, and the error code on failure, but never the SecretString content. If you use a structured logger, ensure the secret value is not included in the event payload. Also avoid embedding secrets in environment variables that get printed during deployment; the whole point of Secrets Manager is to keep values out of configuration files and logs.
Try matching browser tools
All toolsSmall utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Python guides
All Python guidesPython iter(): How the Built-in Function Works
Explains Python's iter() built-in: how it creates iterators, the two-argument sentinel form, custom iterator classes, and common edge cases.
Python Pika: Queues, Exchanges, Routing Keys, and Acknowledgements
Learn how to use Python's Pika client with RabbitMQ: declare queues and exchanges, publish with routing keys, consume messages, and handle acknowledgements reliably.
Python Kafka: kafka-python vs Confluent Client
python kafka python vs confluent kafka: Compare kafka-python and Confluent Kafka Python clients: API differences, performance, feature coverage, and when to choose eac...
Python Kafka Offsets, Partitions, and Manual Commits
Learn how Kafka offsets and partitions work and how to manually commit offsets in Python consumers with kafka-python and confluent-kafka. Covers sync vs async commits, rebalances, and seeking.
More in Python
All Python guidesThe guides filed next to this one in Python, in library order.