I'm sending following REST request to the Read The Docs API:
GET /api/v3/projects/<my_project>/redirects/
I receive JSON in the response body, but I'm unable to parse the JSON into something useful.
I'm using a Python3 program:
import requests
import json
URL = 'https://readthedocs.com/api/v3/projects/<my_project>/redirects/'
TOKEN = "<my_access_token>"
HEADERS = {'Authorization': f'token {TOKEN}'}
response = requests.get(URL, headers=HEADERS, stream=True)
print(f'Response is {response.json()}')
This program returns something that looks like JSON.
If I run it using the following command line:
python3 rtd_get_redirects.py | python3 -m json.tool
I receive the following error:
Expecting value: line 1 column 1 (char 0)
The entire response is large, so I'll just paste this snippet from the beginning:
{'count': 521, 'next': 'https://readthedocs.com/api/v3/projects/signalfx-product-docs/redirects/?limit=10&offset=10', 'previous': None, 'results': [{'_links': {'_self': 'https://readthedocs.com/api/v3/projects/signalfx-product-docs/redirects/6612/', 'project': 'https://readthedocs.com/api/v3/projects/signalfx-product-docs/'}, 'created': '2022-07-20T18:08:52.133088Z',
I haven't found the docs for that use case. How can I get the request body, ensure it's a valid JSON (any valid JSON, including numbers, string, booleans, and nulls, not only objects and arrays) and get the actual JSON.
Using Pydantic forces the JSON to have a specific structure.
You can find nearly everything inside the Request object
You are able to get request body with request.json(), which will give you the parsed JSON as dictionary.
from fastapi import Request, FastAPI
#app.post("/dummypath")
async def get_body(request: Request):
return await request.json()
If you want access the body as string, you can use request.body()
The accepted answer is valid as well, but FastAPI provides a built-in way to do that - check the Singular values in body section in docs.
A parameter with the default Body gets all the payload that doesn't match passed Pydantic-typed parameters (the whole payload in our case) and converts it to the dict. In case of invalid JSON, a standard validation error would be produced.
from fastapi import Body, FastAPI
app = FastAPI()
#app.post('/test')
async def update_item(
payload: dict = Body(...)
):
return payload
UPD: Note on ... (Ellipsis) - it allows marking a value as required. Read more in the Required with Ellipsis docs section
If you are confident that the incoming data is "a valid JSON", you can create a simple type annotation structure to receive the arbitrary JSON data.
from fastapi import FastAPI
from typing import Any, Dict, AnyStr, List, Union
app = FastAPI()
JSONObject = Dict[AnyStr, Any]
JSONArray = List[Any]
JSONStructure = Union[JSONArray, JSONObject]
#app.post("/")
async def root(arbitrary_json: JSONStructure = None):
return {"received_data": arbitrary_json}
Examples
1. JSON object
curl -X POST "http://0.0.0.0:6022/" -H "accept: application/json" -H "Content-Type: application/json" -d "{\"test_key\":\"test_val\"}"
Response:
{
"received_data": {
"test_key": "test_val"
}
}
2. JSON array
curl -X POST "http://0.0.0.0:6022/" -H "accept: application/json" -H "Content-Type: application/json" -d "[\"foo\",\"bar\"]"
Response:
{
"received_data": [
"foo",
"bar"
]
}
If you are not sure about the content type of the incoming data, better to parse the request body.
It can be done as,
from fastapi import FastAPI, Request
app = FastAPI()
#app.post("/")
async def root(request: Request):
return {"received_request_body": await request.body()}
The advantage of this method is that the body will contain any kind of data, JSON, form-data, multipart-form-data, etc.
from fastapi import Request
async def synonyms__select(request: Request):
return await request.json()
will return a JSON object.
This is an example to print the content of a Request, it will print the json body (if it is json parsable) otherwise just print the raw bytes of the body.
async def print_request(request):
print(f'request header : {dict(request.headers.items())}' )
print(f'request query params : {dict(request.query_params.items())}')
try :
print(f'request json : {await request.json()}')
except Exception as err:
# could not parse json
print(f'request body : {await request.body()}')
#app.post("/printREQUEST")
async def create_file(request: Request):
try:
await print_request(request)
return {"status": "OK"}
except Exception as err:
logging.error(f'could not print REQUEST: {err}')
return {"status": "ERR"}
FastAPI has a JSON encoder.
There are some cases where you might need to convert a data type (like
a Pydantic model) to something compatible with JSON
from fastapi import FastAPI
from fastapi.encoders import jsonable_encoder
from pydantic import BaseModel
import simplejson as json
class SubmitGeneral(BaseModel):
controllerIPaddress: str
readerIPaddress: str
ntpServer: str
#app.post("/submitGeneral")
async def submitGeneral(data: SubmitGeneral):
data = jsonable_encoder(data)
#data = json.loads(data.json()) # same as above line
print(f"data = {json.dumps(data)}")
# you have to access the properties with brackets, not by dot notation
query = f"update LocalPLC set ControllerIpAddress = '{data['controllerIPaddress']}', ReaderIPAddress = '{data['readerIPaddress']}'"
return {"status": "OK"}
For those of you using BaseModel and want to have a JSON field, you can import Json from pydantic
from fastapi import FastAPI
from pydantic import BaseModel, Json, Field
app = FastAPI()
class MockEndpoint(BaseModel):
endpoint: str = Field(description="API endpoint to mock")
response: Json = Field(description="Example response of the endpoint")
#app.get("/")
async def root():
return {"message": "Hello World"}
#app.post("/mock")
async def mock_request(mock_endpoint: MockEndpoint):
return mock_endpoint
I am using a python flask server API to call another python flask server API. Both these have been generated using swagger editor.
API2 returns an encoded string such as:
def Server2_API2():
payload = {
"value1": 'somedata',
"value2": 'somedata',
}
encoded_payload = jwt.encode(payload, SECRET, algorithm=ALGORITHM)
return encoded_payload <-- {str}'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.'
API1 on server 1 calls API2 on server 2 as follows:
def Server1_API1():
resp = requests.post('http://localhost:1000/API2', data='value1=one&value2=two')
if resp.status_code == 200:
first_try = resp.content <-- b'"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9"\n'
second_try = resp.content.decode() <-- {str} '"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9"
' <- notice the new line here
return first_try
#return second_try
If I return first_try then I get the error: Object of type 'bytes' is not JSON serializable
If I return second_try then I get the following data returned:
"\"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9\"\n"
My goal is to have just the data returned as follows without any error or any back slashes or new lines:
e.g. "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9"
How can I achieve this?
SOLUTION:
third_try = resp.json() <-- This does the trick :)
This did the trick:
token = resp.json()
I have created an application in the Nexmo dashboard with an event url and answer url. I run the following code I got from Nexmo´s GitHub:
client = nexmo.Client(key=api_key, secret=api_secret, application_id=application_key, private_key=private_key)
response = client.create_call({
'to': [{'type': 'phone', 'number': 'call_to_number'}],
'from': {'type': 'phone', 'number': 'call_from_number'},
'answer_url': ['http://my_event_url']
})
And the phonenumber is called, but nexmo hangs up right away (within a second without saying anything).
On my server I see Nexmo calls the answer url (with the ncco)
what I do when the answer url is called:
import nexmo
import json
from django.http import JsonResponse
#csrf_exempt
def answer(request):
ncco = [{
"action": "talk",
"voiceName": "Amy",
"text": "Thank you for calling Nexmo. Please leave your message after the tone."
}]
d = json.dumps(ncco)
j = is_json(d)
if j:
MyObject.objects.create(message="valid")
else:
MyObject.objects.create(message=user_ip + "json error")
return JsonResponse(d, status=200, safe=False)
def is_json(myjson):
try:
json_object = json.loads(myjson)
except ValueError:
return False
return True
This is what I do when my event_url is called:
#csrf_exempt
def event(request):
d = json.dumps(request.POST)
DataReceived.objects.create(answer=d)
data = {"ok": True}
return JsonResponse(data, status=200)
The event url is called 5 times by Nexmo but the dictionaries are always empty.
`I think you might be "double JSON dumping" your NCCO,
you create the ncco as a python dict, then turn that into a json string with d = json.dumps(ncco) and then you are calling JsonResponse on it, try passing the ncco object to JsonResponse
return JsonResponse(ncco, status=200, safe=False)
I am new to Python and Django. I am an IT professional that deploys software that monitors computers. The api outputs to JSON. I want to create a Django app that reads the api and outputs the data to an html page. Where do I get started? I think the idea is to write the JSON feed to a Django model. Any help/advice is greatly appreciated.
Here's a simple single file to extract the JSON data:
import urllib2
import json
def printResults(data):
theJSON = json.loads(data)
for i in theJSON[""]
def main():
urlData = ""
webUrl = urllib2.urlopen(urlData)
if (webUrl.getcode() == 200):
data = webUrl.read()
printResults(data)
else:
print "Received error"
if __name__ == '__main__':
main()
If you have an URL returning a json as response, you could try this:
import requests
import json
url = 'http://....' # Your api url
response = requests.get(url)
json_response = response.json()
Now json_response is a list containing dicts. Let's suppose you have this structure:
[
{
'code': ABC,
'avg': 14.5,
'max': 30
},
{
'code': XYZ,
'avg': 11.6,
'max': 21
},
...
]
You can iterate over the list and take every dict into a model.
from yourmodels import CurrentModel
...
for obj in json_response:
cm = CurrentModel()
cm.avg = obj['avg']
cm.max = obj['max']
cm.code = obj['code']
cm.save()
Or you could use a bulk method, but keep in mind that bulk_create does not trigger save method.