Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Requests: Reading Response Text, JSON, Headers, and Status Codes

Learn how to use Python requests to read response text, parse JSON, inspect headers, and handle status codes.

~4 min readPython
#requests#HTTP#JSON#API
Illustration of a Python requests response object showing a status code badge, a headers panel, and a JSON document.

When you make an HTTP request with Python's requests library, the returned Response object carries the server's reply. The members you will use most often are response.text, response.json(), response.headers, and response.status_code. Each serves a different purpose, and knowing when to use which one helps you write reliable API clients, web scrapers, and integration scripts.

Reading the Response Body as Text

The response.text property returns the body as a string. Requests infers the character encoding from the Content-Type header, falling back to ISO-8859-1 when no charset is given. This works for most text-based responses, but you may need to override it when the server reports a missing or incorrect charset.

python
import requests response = requests.get("https://api.example.com/data") print(response.text)

If the response contains binary data, such as an image or a compressed file, text will try to decode it and produce garbage. In that case, use response.content to get the raw bytes:

python
image_bytes = requests.get("https://example.com/image.png").content

When you know the response is text but the encoding is wrong, set response.encoding before accessing text:

python
response.encoding = "utf-8" text = response.text

This is useful when the server omits the charset or sends an inaccurate one. The text property is re-evaluated each time you access it, so changing encoding affects subsequent reads.

Parsing JSON from the Response

For JSON APIs, response.json() is the direct way to get a Python dictionary or list. It parses the decoded body with the same JSON engine as json.loads(). If the body is not valid JSON, it raises a JSON decoding error. In recent requests releases, this is requests.exceptions.JSONDecodeError; it is a subclass of ValueError. Catching ValueError covers that case as well as older versions that raise the standard library exception directly.

python
response = requests.get("https://api.example.com/user/1") try: user = response.json() except ValueError: print("Response was not valid JSON") print(response.text)

Unlike some HTTP clients, response.json() does not inspect the Content-Type header before parsing. A response with a missing or non-standard content type can still be valid JSON and parse correctly. You can also parse manually with json.loads(response.text) if you need to pass decoder options directly, but response.json() is simpler for typical API calls.

One common mistake is to call response.json() on a response that returns an empty body or an error page. Always check the status code or wrap the call in a try-except block to avoid crashing the script.

Inspecting Response Headers

response.headers returns a CaseInsensitiveDict, which means you can access header names without worrying about capitalization. This is convenient because HTTP header names are case-insensitive by spec, but many servers and proxies send them in mixed case.

python
response = requests.get("https://api.example.com/data") content_type = response.headers["Content-Type"] # or content_type = response.headers.get("content-type")

Common headers you might inspect include Content-Type, Content-Length, Cache-Control, ETag, and RateLimit-Remaining. The headers object behaves like a dictionary, so you can iterate over it or use .get() to avoid KeyError.

python
for name, value in response.headers.items(): print(f"{name}: {value}")

Note that response.headers contains only the response headers, not the request headers. If you need to debug the request side, use response.request.headers.

Checking Status Codes

The status_code attribute reports the HTTP status of the final response. A status code in the 2xx range means success, 3xx means redirect, 4xx is a client error, and 5xx is a server error. Requests follows redirects by default, so the status_code you inspect is the final response after any redirects.

python
response = requests.get("https://api.example.com/data") if response.status_code == 200: data = response.json() elif response.status_code == 404: print("Resource not found") else: print(f"Unexpected status: {response.status_code}")

A more concise way is to use response.raise_for_status(). This method raises requests.exceptions.HTTPError if the status code indicates an error (4xx or 5xx). It is a good practice to call it after every request unless you explicitly want to handle non-2xx responses yourself.

python
try: response = requests.get("https://api.example.com/data") response.raise_for_status() data = response.json() except requests.exceptions.HTTPError as err: print(f"HTTP error: {err}")

Be aware that raise_for_status() does not check the response body; it only looks at the status code. A 200 response with a malformed JSON body will not raise an HTTPError, but response.json() will raise a JSON decode error.

Handling Errors and Unexpected Responses

In real-world API integration, you need to handle both network-level failures and application-level errors. The requests library raises exceptions for connection problems, timeouts, and too many redirects. These are separate from HTTP status errors.

python
import requests try: response = requests.get("https://api.example.com/data", timeout=5) response.raise_for_status() data = response.json() except requests.exceptions.Timeout: print("The request timed out") except requests.exceptions.ConnectionError: print("Could not connect to the server") except requests.exceptions.HTTPError as err: print(f"HTTP error: {err}") except ValueError: print("Response was not valid JSON")

When you expect a specific content type, you can inspect it before parsing, but do not treat an unusual Content-Type as proof that the body is not JSON. Some APIs return JSON with types such as text/plain or application/hal+json. If you want to be tolerant of those responses, call response.json() and handle ValueError rather than rejecting the response based on an exact header match.

Performance and Resource Considerations

When dealing with large responses, the default behavior of requests is to download the entire body into memory. This is fine for most API calls, but for large files or streaming endpoints, you should use stream=True and iterate over the content in chunks.

python
with requests.get("https://example.com/large-file", stream=True) as response: for chunk in response.iter_content(chunk_size=8192): # handle the chunk pass

Using a context manager (with) ensures the connection is released back to the pool after the block exits. Without it, you should call response.close() manually, especially when streaming.

For JSON responses, response.json() loads the entire body into memory as a Python object. If the JSON is extremely large, consider parsing it incrementally with json.JSONDecoder.raw_decode() on a streaming response, but this is rarely necessary for typical API usage.

Another resource consideration is connection pooling. The requests library reuses connections through a Session object. If you make many requests to the same host, create a Session and reuse it to avoid the overhead of establishing a new TCP connection each time.

python
session = requests.Session() response1 = session.get("https://api.example.com/endpoint1") response2 = session.get("https://api.example.com/endpoint2")

Finally, always set a timeout on your requests. Without a timeout, a hanging server can block your script indefinitely. A short timeout like 5 seconds is often enough for internal APIs, while public endpoints may need longer.

python
try: response = requests.get("https://api.example.com", timeout=5) response.raise_for_status() except requests.exceptions.Timeout: print("Request timed out")

By combining status_code, headers, text, and json(), you can build robust HTTP clients that handle both expected and unexpected server behavior. The key is to check the status code when needed, understand what the body contains, and parse the body only when it is safe to do so.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.