REST API using Python requests module
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
You can interact with a REST API endpoint using Python request module and today we will discuss how to make different REST API requests like HEAD, GET, POST, PUT, PATCH, and DELETE using Python requests module.
If you wonder what REST API is, you can refer this blog post where I have given all the details to clear your doubts.
REST API using Python requests module
Prerequisite
- One Linux system with python, pip and docker installed
- Python request module installed
Setup:
We will use a docker container that has REST API application pre-configured for this demo. Login to your Linux shell and execute below command to launch a new docker container.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
## ----- ## Setup ## ----- ## Install docker https://docs.docker.com/engine/install/ ## Install python and pip ## Install python request moduled ## Start sample rest api application docker container run --name myAPI --publish 8080:8080 --detach ericgoebelbecker/resttutorial |
REST API HTTP HEAD request example using Python requests module:
The HTTP HEAD method requests the headers that would be returned if the HEAD request’s URL was instead requested with the HTTP GET method. For example, if a URL might produce a large download, a HEAD request could read its Content-Length header to check the file size without actually downloading the file.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
## ------------ ## HEAD request ## ------------ ## Create a python script cat <<'EOF'> rest.py import requests headers = {'Accept': 'application/json', 'Content-Type': 'application/json'} data = requests.head('http://127.0.0.1:8080/api/tutorial/1.0/employees', headers=headers) print(data) print(data.headers) EOF ## Execute the script python rest.py ## Returns ## ## {'Transfer-Encoding': 'chunked', 'Date': 'Wed, 05 Jan 2022 08:30:41 GMT', ## 'Content-Type': 'application/json;charset=UTF-8'} |
REST API HTTP GET request using Python requests module:
GET method is used to retrieve resource information or to get information from a REST API endpoint. If the request is success, the endpoint response with a status code of 200 along with the requested data and if failed, response will contain the respective status codes like 404 or 400 etc.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
## ----------- ## GET request ## ----------- ## Get all employees data ## Create a python script cat <<'EOF'> get_all.p import requests import json headers = {'Accept': 'application/json', 'Content-Type': 'application/json'} data = requests.get('http://127.0.0.1:8080/api/tutorial/1.0/employees', headers=headers) data_json = json.loads(data.text) print (json.dumps(data_json, indent=2)) EOF ## Execute the script python get_all.py ## Returns ## [ ## { ## "lastName": "Doe", ## "phone": "555-1212", ## "employeeId": 1, ## "firstName": "John", ## "email": "john@doe.com" ## }, ## { ## "lastName": "Doe", ## "phone": "867-5309", ## "employeeId": 2, ## "firstName": "Jenny", ## "email": "jenny@doe.com" ## }, ## { ## "lastName": "Kent", ## "phone": "555-1213", ## "employeeId": 3, ## "firstName": "Clark", ## "email": "clark@doe.com" ## } ## ] ## Get specific employee data ## Create a python script cat <<'EOF'> get_one.py import requests import json import sys id = sys.argv[1] url = 'http://127.0.0.1:8080/api/tutorial/1.0/employees/'+str(id) headers = {'Accept': 'application/json', 'Content-Type': 'application/json'} data = requests.get(url, headers=headers) data_json = json.loads(data.text) print (json.dumps(data_json, indent=2)) EOF ## Execute the script python get_one.py 1 ## Returns ## { ## "lastName": "Doe", ## "phone": "555-1212", ## "employeeId": 1, ## "firstName": "John", ## "email": "john@doe.com" ## } |
Print REST API HTTP GET request and response details using Python requests module
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
## Get entire request and responce along with data ## Create a python script ## pip install requests_toolbelt cat <<'EOF'> rest.py import requests from requests_toolbelt.utils import dump def logging_hook(response, *args, **kwargs): data = dump.dump_all(response) print(data.decode('utf-8')) http = requests.Session() http.hooks["response"] = [logging_hook] http.get("http://127.0.0.1:8080/api/tutorial/1.0/employees/1") EOF ## Execute the script python rest.py ## Returns ## < GET /api/tutorial/1.0/employees/1 HTTP/1.1 ## < Host: 127.0.0.1:8080 ## < Connection: keep-alive ## < Accept-Encoding: gzip, deflate ## < Accept: */* ## < User-Agent: python-requests/2.27.0 ## < ## ## > HTTP/1.1 200 ## > Content-Type: application/json;charset=UTF-8 ## > Transfer-Encoding: chunked ## > Date: Wed, 05 Jan 2022 08:39:39 GMT ## > ## {"employeeId":1,"firstName":"John","lastName":"Doe","email":"john@doe.com","phone":"555-1212"} |
REST API HTTP POST request with data from a file using Python requests module:
POST method is used to create a new resource. If the resource is created successfully the response will contain status code 201 and if failed, response will contain the respective status codes like 404 or 409 etc.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
## ------------ ## POST request ## ------------ ## Using data from a file ## Create a new file with employee details cat <<'EOF'> employee.json { "employeeId": 4, "firstName": "Debjeet", "lastName": "Bhowmik", "email": "debjeettoni@gmail.com", "phone": "9876543210" } EOF ## Create a python script cat <<'EOF'> rest.py import requests import json headers = {'Accept': 'application/json', 'Content-Type': 'application/json'} with open('employee.json', 'rb') as payload: data = requests.post('http://127.0.0.1:8080/api/tutorial/1.0/employees', data=payload, headers=headers) print(data) payload.close() EOF ## Execute the script python rest.py ## Check if new employee created python get_one.py 4 ## Returns ## { ## "lastName": "Bhowmik", ## "phone": "9876543210", ## "employeeId": 4, ## "firstName": "Debjeet", ## "email": "debjeettoni@gmail.com" ## } |
REST API HTTP POST request with data from script using Python requests module:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
## Using data from the script ## Create a python script cat <<'EOF'> rest.py import requests import json headers = {'Accept': 'application/json', 'Content-Type': 'application/json'} responce = requests.post('http://127.0.0.1:8080/api/tutorial/1.0/employees', json={ "employeeId": 5, "firstName": "Chandrima", "lastName": "Koley", "email": "chandrima@gmail.com", "phone": "8253288366" }, headers=headers) print(responce) EOF ## Execute the script python rest.py ## Check if new employee created python get_one.py 5 ## Returns ## { ## "lastName": "Koley", ## "phone": "8253288366", ## "employeeId": 5, ## "firstName": "Chandrima", ## "email": "chandrima@gmail.com" ## } |
How to use environment variable in data of Python requests module?
So far, we have learned how to POST with data from file and from script using Python requests module command. But sometimes you may want to pass the data dynamically using Linux environment variable in our Python script.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
## Using data from the script with environment variables ## Create a new env variable export PHONE=9999999999 source ~/.bashrc ## Create a python script cat <<'EOF'> rest.py import requests import json import os phone_no = os.environ['PHONE'] headers = {'Accept': 'application/json', 'Content-Type': 'application/json'} responce = requests.post('http://127.0.0.1:8080/api/tutorial/1.0/employees', json={ "employeeId": 6, "firstName": "Avik", "lastName": "Dutta", "email": "avik@gmail.com", "phone": phone_no }, headers=headers) print(responce) EOF ## Execute the script python rest.py ## Check if new employee created python get_one.py 6 ## Returns ## { ## "lastName": "Dutta", ## "phone": "9999999999", ## "employeeId": 6, ## "firstName": "Avik", ## "email": "avik@gmail.com" ## } |
Next, we will discuss how to update an existing resource using PUT request and different between HTTP PUT and PATCH methods.
REST API HTTP PUT request using Python requests module:
PUT method is used to update or replace an existing resource. If the resource is updated successfully the response will contains status code 200 and if failed, response will contain the respective status codes like 404 or 204 etc.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
## ----------- ## PUT request ## ----------- ## Create a python script cat <<'EOF'> rest.py import requests import json headers = {'Accept': 'application/json', 'Content-Type': 'application/json'} responce = requests.put('http://127.0.0.1:8080/api/tutorial/1.0/employees/6', json={ "employeeId": 6, "firstName": "Senha", "lastName": "Kurin", "email": "skurin@gmail.com", "phone": "1111111111" }, headers=headers) print(responce) EOF ## Execute the script python rest.py ## Check if new employee created python get_one.py 6 ## Returns ## { ## "lastName": "Kurin", ## "phone": "1111111111", ## "employeeId": 6, ## "firstName": "Senha", ## "email": "skurin@gmail.com" ## } |
REST API HTTP PATCH request using Python requests module:
PATCH is similar to PUT request but can update a subset of the resource without replacing the entire content. For example, if you want to update a specific field in your data then use PATCH request and if you want to update the entire content then use PUT request. If the resource is updated successfully the response will contains status code 200 and if failed, response will contain the respective status codes like 404 or 204 etc.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
## ------------- ## PATCH request ## ------------- ## Create a python script cat <<'EOF'> rest.py import requests import json headers = {'Accept': 'application/json', 'Content-Type': 'application/json'} responce = requests.patch('http://127.0.0.1:8080/api/tutorial/1.0/employees/6', json={"phone": "0123456789"}, headers=headers) print(responce) EOF ## Execute the script python rest.py ## Check if new employee created python get_one.py 6 ## Returns ## { ## "lastName": "Kurin", ## "phone": "0123456789", ## "employeeId": 6, ## "firstName": "Senha", ## "email": "skurin@gmail.com" ## } |
REST API HTTP DELETE request using Python requests module:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
## -------------- ## DELETE request ## -------------- ## Create a python script cat <<'EOF'> rest.py import requests import json headers = {'Accept': 'application/json', 'Content-Type': 'application/json'} responce = requests.delete('http://127.0.0.1:8080/api/tutorial/1.0/employees/6', headers=headers) print(responce) EOF ## Execute the script python rest.py ## Check if employee deleted python get_all.py ## Returns ## [ ## { ## "lastName": "Doe", ## "phone": "555-1212", ## "employeeId": 1, ## "firstName": "John", ## "email": "john@doe.com" ## }, ## { ## "lastName": "Doe", ## "phone": "867-5309", ## "employeeId": 2, ## "firstName": "Jenny", ## "email": "jenny@doe.com" ## }, ## { ## "lastName": "Kent", ## "phone": "555-1213", ## "employeeId": 3, ## "firstName": "Clark", ## "email": "clark@doe.com" ## }, ## { ## "lastName": "Bhowmik", ## "phone": "9876543210", ## "employeeId": 4, ## "firstName": "Debjeet", ## "email": "debjeettoni@gmail.com" ## }, ## { ## "lastName": "Koley", ## "phone": "8253288366", ## "employeeId": 5, ## "firstName": "Chandrima", ## "email": "chandrima@gmail.com" ## } ## ] |
How to authorize REST API requests in Python requests module command?
If your REST API endpoint has some authorization mechanism enabled (most have) then you need to authenticate your HTTP request with different authorization schema.
Basic Auth:
It is a simple authentication scheme built into the HTTP protocol. The client sends HTTP requests with the Authorization header that contains the word Basic, followed by a space and a base64-encoded(non-encrypted) string username: password. For example, to authorize as username / Pa$$w0rd the client would send.
Bearer Token:
Commonly known as token authentication. It is an HTTP authentication scheme that involves security tokens called bearer tokens. As the name depicts “Bearer Authentication” gives access to the bearer of this token.
API Key:
An API key is a token that a client provides when making API calls. With API key auth, you send a key-value pair to the API either in the request headers or query parameters. Some APIs use API keys for authorization.
Digest Auth:
Digest Authentication communicates credentials in an encrypted form by applying a hash algorithm to the username and the password, the password is converted to response and then it is sent to the server. After that server supplies nonce value, the HTTP method, and the requested URI.
OAuth:
OAuth permits client applications to access data provided by a third-party API. For example, as a user of a service you can grant another application access to your data with that service without exposing your login details.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
## -------------- ## Authentication ## -------------- ## Basic Authentication import requests requests. #or import requests from requests.auth import HTTPBasicAuth requests. ## Token import requests headers={ 'Authorization': 'Bearer requests. ## Oauth import requests from requests_oauthlib import OAuth1 auth = OAuth1('YOUR_APP_KEY', 'YOUR_APP_SECRET', 'USER_OAUTH_TOKEN', 'USER_OAUTH_TOKEN_SECRET') requests. |
Clean up:
|
1 2 3 4 5 6 7 8 |
## -------- ## Clean up ## -------- ## Remove the docker container and image docker container stop myAPI docker container rm myAPI docker image rm docker.io/ericgoebelbecker/resttutorial |