Python Boto3 S3: Upload, Download, List, and Delete Files
Learn how to upload, download, list, and delete objects in AWS S3 using Python boto3, with code examples, error handling, and performance and security considerations.

Working with files in AWS S3 from Python usually means using boto3. The core operations—upload, download, list, and delete—are straightforward once you know the available methods and their parameters. This article shows how to perform those operations with boto3, including code examples, error handling, and production considerations.
Prerequisites and Setup
Before any S3 operation, install boto3 and configure AWS credentials:
pip install boto3Credentials can come from environment variables, the shared AWS configuration files, or an IAM role. For local development, put access keys in ~/.aws/credentials and set the region in ~/.aws/config:
# ~/.aws/credentials
[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY# ~/.aws/config
[default]
region = us-east-1Boto3 offers two main interfaces: the low-level client and the high-level resource. The client mirrors the AWS API directly, while the resource provides a more Pythonic object model. Most file operations work well with either interface. The examples below use the client unless noted, because it exposes the full set of parameters and is often more direct for fine-grained control.
import boto3
s3_client = boto3.client('s3')
s3_resource = boto3.resource('s3')Uploading Files to S3
Uploading a file from your local filesystem to an S3 bucket is done with upload_file. This method handles the underlying HTTP request and, for larger files, automatically uses multipart upload.
s3_client.upload_file(
Filename='local/path/to/file.txt',
Bucket='my-bucket',
Key='remote/path/file.txt'
)The Filename is the local path, Bucket is the bucket name, and Key is the destination object key. The key can include slashes to simulate folders, but S3 storage is flat—folders are just key prefixes.
If you already have a file-like object in memory, use upload_fileobj. This is useful when the data comes from a network stream or a BytesIO buffer.
import io
buffer = io.BytesIO(b'data to upload')
s3_client.upload_fileobj(buffer, 'my-bucket', 'data.txt')For small payloads that are already in memory, put_object is simpler, but it does not support multipart upload and has a 5 GB limit.
s3_client.put_object(Bucket='my-bucket', Key='hello.txt', Body=b'Hello, S3!')When uploading, you can set metadata, access control, and storage class through ExtraArgs. On buckets with ACLs enabled, you can make an object publicly readable with:
s3_client.upload_file(
Filename='file.txt',
Bucket='my-bucket',
Key='file.txt',
ExtraArgs={'ACL': 'public-read'}
)Downloading Files from S3
The counterpart to upload_file is download_file, which saves an object directly to a local file.
s3_client.download_file(
Bucket='my-bucket',
Key='remote/path/file.txt',
Filename='local/path/file.txt'
)If you need the object content in memory, use get_object and read the Body stream.
response = s3_client.get_object(Bucket='my-bucket', Key='file.txt')
content = response['Body'].read().decode('utf-8')For streaming downloads, download_fileobj writes to a file-like object.
with open('output.bin', 'wb') as f:
s3_client.download_fileobj(Bucket='my-bucket', Key='data.bin', Fileobj=f)When downloading, you can use VersionId to retrieve a specific version if versioning is enabled on the bucket.
Listing Objects in an S3 Bucket
Listing objects requires pagination because S3 returns at most 1,000 keys per request. The list_objects_v2 method is the current API and supports a ContinuationToken for pagination.
def list_all_objects(bucket_name, prefix=''):
paginator = s3_client.get_paginator('list_objects_v2')
for page in paginator.paginate(Bucket=bucket_name, Prefix=prefix):
for obj in page.get('Contents', []):
yield obj['Key']The Prefix parameter filters results to keys that start with a given string. This is how you simulate listing a "folder"—you use the folder path as the prefix.
If you need only object names, you can iterate directly over the Contents list. Each object also includes metadata like Size, LastModified, and ETag.
Deleting Files from S3
To delete a single object, use delete_object.
s3_client.delete_object(Bucket='my-bucket', Key='file.txt')For multiple objects, delete_objects accepts a list of keys and performs the deletion in a single request. This is more efficient than looping over delete_object calls.
objects_to_delete = [{'Key': 'file1.txt'}, {'Key': 'file2.txt'}]
response = s3_client.delete_objects(
Bucket='my-bucket',
Delete={'Objects': objects_to_delete}
)The response includes any errors that occurred, so you should check Errors in the response for partial failures.
Deleting an object that does not exist does not raise an error; S3 returns a 204 No Content. This is important for idempotent cleanup scripts.
Error Handling and Retries
S3 operations can fail for many reasons: network issues, invalid credentials, missing buckets, or permission errors. Boto3 raises ClientError for most service-side failures. The exception's response includes an error code and message that you can inspect.
from botocore.exceptions import ClientError
try:
s3_client.get_object(Bucket='my-bucket', Key='nonexistent.txt')
except ClientError as e:
error_code = e.response['Error']['Code']
if error_code == 'NoSuchKey':
print('Object not found')
elif error_code == 'AccessDenied':
print('Permission denied')
else:
print(f'Unexpected error: {error_code}')Boto3 retries certain transient errors, such as throttling and network timeouts, using the SDK's configured retry behavior. You can adjust retry settings via the Config parameter when creating the client.
from botocore.config import Config
config = Config(retries={'max_attempts': 5, 'mode': 'standard'})
s3_client = boto3.client('s3', config=config)Performance and Security Considerations
For files above the configured threshold, upload_file uses multipart upload, which parallelizes the transfer. The default threshold is 8 MB, and you can tune it with multipart_threshold and max_concurrency in TransferConfig.
from boto3.s3.transfer import TransferConfig
config = TransferConfig(multipart_threshold=16 * 1024 * 1024, max_concurrency=10)
s3_client.upload_file(
Filename='large.bin',
Bucket='my-bucket',
Key='large.bin',
Config=config
)On the security side, always use IAM roles or least-privilege policies. For example, a policy that allows only s3:GetObject on a specific bucket prevents accidental deletion. For sensitive data, enable server-side encryption on the bucket or set the ServerSideEncryption parameter when uploading.
s3_client.put_object(
Bucket='my-bucket',
Key='secret.txt',
Body=b'data',
ServerSideEncryption='AES256'
)Choosing Between Client and Resource for File Operations
The resource interface can be more readable for simple tasks. For example, uploading with the resource:
bucket = s3_resource.Bucket('my-bucket')
bucket.upload_file('local.txt', 'remote.txt')The client is the lower-level interface and maps directly to the S3 API. It exposes explicit pagination, batch operations, and error details. The resource is convenient for common single-object operations, but it hides some of those details. In a script that combines many operations, using one interface consistently keeps the code predictable.
Both interfaces are valid. For most production scripts, the client is often the more straightforward choice because the API surface is visible. The resource is fine for prototypes or when you prefer object-oriented syntax.
Try matching browser tools
All toolsSmall utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Python guides
All Python guidesPython iter(): How the Built-in Function Works
Explains Python's iter() built-in: how it creates iterators, the two-argument sentinel form, custom iterator classes, and common edge cases.
Python Pika: Queues, Exchanges, Routing Keys, and Acknowledgements
Learn how to use Python's Pika client with RabbitMQ: declare queues and exchanges, publish with routing keys, consume messages, and handle acknowledgements reliably.
Python Kafka: kafka-python vs Confluent Client
python kafka python vs confluent kafka: Compare kafka-python and Confluent Kafka Python clients: API differences, performance, feature coverage, and when to choose eac...
Python Kafka Offsets, Partitions, and Manual Commits
Learn how Kafka offsets and partitions work and how to manually commit offsets in Python consumers with kafka-python and confluent-kafka. Covers sync vs async commits, rebalances, and seeking.
More in Python
All Python guidesThe guides filed next to this one in Python, in library order.