Python OpenAI File Upload: How to Upload Files with the API
Learn how to upload files to the OpenAI API with Python using files.create, including parameters, error handling, and use in fine-tuning and assistants.

Uploading files to the OpenAI API is a common task when you need to provide training data for fine-tuning, attach documents to an assistant, or use the Files API for other workflows. In Python, the openai package exposes client.files.create, which handles multipart uploads. This article covers the practical details: setup, parameters, error handling, and how to use the returned file ID in later API calls.
Prerequisites and Setup
Before you can upload a file, install the openai Python package and configure an API key. The package is available on PyPI and can be installed with pip:
pip install openaiUse a Python version supported by the SDK; the exact minimum depends on the package release. After installation, create a client instance with your API key. Avoid hardcoding the key in source code; use an environment variable instead:
export OPENAI_API_KEY='your-api-key'Then in your Python code:
import os
from openai import OpenAI
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))If the key is missing or invalid, the client raises an OpenAIError. For local development, you can use a .env file with the python-dotenv package, but the environment variable approach is the most portable.
Basic File Upload with files.create
The core method for uploading a file is client.files.create. It expects a file object opened in binary mode and a purpose string that tells the API how the file will be used. Here is a minimal example:
from openai import OpenAI
client = OpenAI() # reads OPENAI_API_KEY from environment
with open('training_data.jsonl', 'rb') as f:
response = client.files.create(
file=f,
purpose='fine-tune'
)
print(response.id)The response is a FileObject with fields such as id, purpose, filename, bytes, and created_at. The id is the value you use later when creating a fine-tuning job or attaching the file to an assistant message.
If you are uploading a file for an assistant, change purpose to 'assistants'. The API accepts a limited set of purposes, and an invalid purpose returns a 400 error.
Understanding File Upload Parameters
files.create accepts several parameters beyond the mandatory file and purpose:
| Parameter | Type | Description |
|---|---|---|
file | file | A file-like object opened in binary mode. Required. |
purpose | string | The intended use: 'fine-tune', 'assistants', or 'batch'. Required. |
filename | string | Optional. Overrides the filename sent to the API. Useful when the local file name is not descriptive. |
The filename parameter is also useful for in-memory objects. For example, you can build an in-memory JSONL object and set a custom name:
import io
import json
record = {'prompt': 'What is AI?', 'completion': 'Artificial Intelligence'}
file_obj = io.BytesIO((json.dumps(record) + '\n').encode())
response = client.files.create(
file=file_obj,
purpose='fine-tune',
filename='custom_name.jsonl'
)Note that the API enforces file size limits and supported file types. The exact limits change over time, so check the official OpenAI documentation for current values. The SDK does not validate these content constraints locally; invalid uploads are rejected by the server.
Handling Upload Errors and Retries
Network failures, rate limits, and invalid inputs can cause uploads to fail. The openai library raises exceptions that you can catch and handle explicitly. Common exception classes include:
openai.APIError: base class for API errors.openai.APIConnectionError: network issues, such as a dropped connection.openai.RateLimitError: exceeded allowed request rate.openai.AuthenticationError: invalid API key.openai.BadRequestError: invalid file or parameters.
A robust upload function should retry transient errors and surface permanent ones. Here is an example that retries connection errors and rate limits with exponential backoff:
import time
from openai import OpenAI, APIConnectionError, RateLimitError
client = OpenAI()
def upload_file(file_path, purpose, max_retries=3):
for attempt in range(max_retries):
try:
with open(file_path, 'rb') as f:
return client.files.create(file=f, purpose=purpose)
except (APIConnectionError, RateLimitError) as e:
if attempt == max_retries - 1:
raise
wait_time = 2 ** attempt
print(f'Upload failed, retrying in {wait_time}s: {e}')
time.sleep(wait_time)For permanent errors such as BadRequestError, fix the request before retrying; retrying the same invalid request will not help. Inspect the exception message to understand what failed—it often includes the field that failed validation.
Using Uploaded Files in Fine-Tuning and Assistants
The file ID returned by files.create is the key to using the file in other API calls. For fine-tuning, pass it to fine_tuning.jobs.create:
# response is the FileObject returned by files.create
job = client.fine_tuning.jobs.create(
training_file=response.id,
model='gpt-3.5-turbo'
)For assistants, attach the file to a message using attachments. The exact syntax can vary by SDK/API version, but a common pattern is:
message = client.beta.threads.messages.create(
thread_id=thread.id,
role='user',
content='Please analyze this document.',
attachments=[{'file_id': response.id, 'tools': [{'type': 'file_search'}]}]
)In both cases, the file must be uploaded with the correct purpose. A file uploaded with purpose='fine-tune' cannot be attached to an assistant message, and vice versa. If you need to use the same data for both, upload it twice with different purposes.
The same pattern applies to Batch API jobs: upload with purpose='batch' and then pass the returned file ID when creating a batch job.
Performance and Operational Considerations
Uploading large files can take time and may use significant memory because the file is sent as a multipart upload. If you regularly hit size limits, preprocess or split files before uploading. The API has a maximum file size, and exceeding it results in a 400 error. Check the current limit in the docs and consider checking the size locally before uploading:
import os
file_size = os.path.getsize('training_data.jsonl')
if file_size > MAX_ALLOWED_SIZE: # check current limit from docs
raise ValueError('File too large')For many uploads, be mindful of rate limits. If you need to upload multiple files, use a semaphore or worker pool to limit concurrency and include retry logic. The SDK also supports automatic retries through a max_retries client parameter, but application-level handling gives you control over backoff.
Security and API Key Management
Never expose your API key in client-side code or commit it to version control. Use environment variables or a secrets manager. Before uploading sensitive data, review OpenAI's data usage and retention policies and your own compliance requirements.
Common Pitfalls and Edge Cases
One frequent mistake is opening the file in text mode instead of binary. The API expects a binary file object; passing a text-mode file can cause encoding errors or corrupted uploads. Always use 'rb'.
Another issue is using an invalid purpose string. The API is strict about allowed values. If you are unsure, check the documentation or the error message returned by the server.
Finally, file IDs can become invalid if you delete them or if API retention rules remove them. Keep a local copy of the file and a record of its ID. If an ID is no longer valid, re-upload the file before using it.
Handling these edge cases helps keep file uploads reliable in production. files.create is straightforward, but the surrounding details—parameter validation, error handling, and operational constraints—determine whether your integration is robust.
Try matching browser tools
All toolsSmall utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Python guides
All Python guidesPython iter(): How the Built-in Function Works
Explains Python's iter() built-in: how it creates iterators, the two-argument sentinel form, custom iterator classes, and common edge cases.
Python Pika: Queues, Exchanges, Routing Keys, and Acknowledgements
Learn how to use Python's Pika client with RabbitMQ: declare queues and exchanges, publish with routing keys, consume messages, and handle acknowledgements reliably.
Python Kafka: kafka-python vs Confluent Client
python kafka python vs confluent kafka: Compare kafka-python and Confluent Kafka Python clients: API differences, performance, feature coverage, and when to choose eac...
Python Kafka Offsets, Partitions, and Manual Commits
Learn how Kafka offsets and partitions work and how to manually commit offsets in Python consumers with kafka-python and confluent-kafka. Covers sync vs async commits, rebalances, and seeking.
More in Python
All Python guidesThe guides filed next to this one in Python, in library order.