Python requests - JSONDecodeError("Expecting value", s, err.value) from None - json

I'm trying to grab a response from an api but i cannot load it in a dictionary.
Here's my code:
r = session.post(url, headers=headers, json=payload)
The returned status code is 200 so i get a response.
By trying to do: reply = r.json() i get:
Traceback (most recent call last):
File "C:/Users/PycharmProjects/honeymonitor/main.py", line 46, in <module>
data = get_data(session)
File "C:/Users/PycharmProjects/honeymonitor/main.py", line 35, in get_data
reply = r.json()
File "C:\Users\PycharmProjects\honeymonitor\venv\lib\site-packages\requests\models.py", line 897, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\AppData\Local\Programs\Python\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
And by using reply = r.text i get a lot of gibberish:
� d�M���)b�b��$�j�np������X2�Al�w{g��wO�n-���w��g�����}�1�Y5#�R��/�
� X���o]>6,�N�_#�#Ң����K�#���s
��Ɩ�ː�����2��N�:D��p�����7{K��������V��
-")�p0n#I�f!���Q�+| /We�.�p�=����u���-�ʌ�cs�"jfH����}qkː��q�,q�bږq�ٟ��Z���>��U#�����ET�V/��+cU����ʁ�LLe��(���V�2���å}�UC���#�8y��B�7Λ��c���Y?��q��TOMTv{����ߎ������,�[�|��wf_�˰�g������?
The content type in the headers of the post request is:
'Content-Type': 'application/json;charset=UTF-8',
Also by checking the r.encoding i get None
At the end i've tried json.loads(r.text) but i get the same error.
Also tried encoding the response with r.content.decode('utf-8')
and got UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 1: invalid continuation byte
What i'm doing wrong?
EDIT:
I've tried replicating the request with Fiddler and i can clearly see the response json there
I've compared the response headers between Fiddler and requests, the only difference is that with requests i ge 'Transfer-Encoding': 'chunked' in the headers
EDIT2:
Here's a portion of the r.content:
b'\x03\xd5\x01\x00d\xca9\x9cvwC\x7f \x1b]j\xfb\xc5\xa3tA\xa1k.N9\x01\x08\x03\r\xcc\x02i\x01\xb7\x0b\xc2=\xc6\x92\xb1\rb\xe3\xbf\xdb;\x83\xb7\xbf{\xe2vk\xd1\xed\xef\xbf[d<{.n\x7fo\xd5WC\xab\x84\xeb\xecN\xd3#\xcc\xed\x0cAks\xa5\xe6\x87?\x15\xb3:\x8d\xfdS}\xb5O\x9a\xb8\x9d\xddK\x98\xf5+HSLq\x14\x84\xab:Fd!\xa8\xc0\x0c\xd2\xd3\x0eL\xd4\xec\x9b\xcb\xf2\xcf\xae\x18\xd9\x15&fg\xbf\xd6.```

There can be a couple of reasons for this error
I faced the same error. In my case, I am calling a put URL. In your case, it is post
I implemented my put API in such a way that, it won't return anything except status code. So after receiving the Response body contains no data. And we are trying to parse that to JSON. But actually nothing is there to parse.

Related

Receive API body POST by Python

I have a problem with API POST, can you help me?
import requests
import json
url = 'http://xxx/api/getTotalPrice'
param = dict(itineraryType=1,
departureAirportCode='HAN',
destinationAirportCode='DLI',
departureDate='2020-12-30T14:00',
returnDate='2020-12-30T09:00',
adult=1,
children=1,
infant=1
)
resp = requests.post(url=url, params=param)
u_data = resp.json()
print(u_data)
I want to receive data from API POST. This is body API and this made error.
Here, this is error
C:\Users\Admin\PycharmProjects\untitled\venv\Scripts\python.exe C:/Users/Admin/PycharmProjects/untitled/venv/test.py
Traceback (most recent call last):
File "C:\Users\Admin\PycharmProjects\untitled\venv\test.py", line 17, in <module>
u_data = resp.json()
File "C:\Users\Admin\PycharmProjects\untitled\venv\lib\site-packages\requests\models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Admin\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Process finished with exit code 1
I can get data from POST API without body, but with body, I cannot.
Can you help to fix code?
Many thanks!
P/S:
Here value which I want to get
{
"departureFlight": {
"totalPrice": 1896000.0,
"airlineCode": "VN"
},
"returnFlight": {
"totalPrice": 3263800.0,
"airlineCode": "VJ"
}
}
OK, I have an answer.
data = {'itineraryType':1,'departureAirportCode':'HAN','destinationAirportCode':'DLI','departureDate':'2020-12-30T15:00','returnDate':'2020-12-30T09:00','adult':1,'children':1,'infant':1}
headers={'Content-type':'application/json', 'Accept':'application/json'}
resp = requests.post(url=url, json=data, headers=headers)
u_data = resp.json()
print(u_data)
change dict(...) to {...} and set headers.

JSONDecodeError: Expecting value: line 1 column 1 error in AWS Lambda

I am trying to write a aws lambda function which will push the SQS queue output in a s3 bucket.
But the lambda function is failing to push the message , the cloudwatch log is showing
JSONDecodeError: Expecting value: line 1 column 1
i am posting the lambda function which i am using
import json
import boto3
def lambda_handler(event, context):
s3 = boto3.client("s3")
data = json.loads(event["Records"][0]["body"]) --getting error in this line
print(data)
s3.put_object(Bucket="sqsmybucket",key="data.json", Body=json.dumps(data))
#print(event)
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
The cloud watch log is showing
2020-05-30T23:51:45.276+05:30
[ERROR] JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 6, in lambda_handler
data = json.loads(event["Records"][0]["body"])
File "/var/lang/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/var/lang/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/var/lang/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
I have formatted the message and saved it to the cloud below is the link
[formatted JSON code][1]
formatted JSON
Please help , thanks in advance
Your event["Records"][0]["body"] is a plain string, not json:
"body": "A difficult message."
Therefore, json.loads(event["Records"][0]["body"]) is equivalent to json.loads("A difficult message.") which obviously fails.
To get body's value you can do the following instead:
data = event["Records"][0]["body"]
However, since later you have the following statment:
Body=json.dumps(data)
The Body will be:
Body='"A difficult message."'
which may or may not be what you desire.

App randomly crashes with a JSON error

I've posted first the error message with full traceback and the code as well.
The code that is below is within a loop, that will run multiple times changing the variables within the session.get() link for the API data call.
From time to time I will get an error that the json.decoder.JSONDecodeError is expecting value... (full error below)
I can't figure out why.
Traceback (most recent call last):
File "C:\Users\CarlosMiguel\Desktop\experiments\data_collector\data_collector.py", line 100, in <module>
get_data = session.get(api_data_link+symbol+"?limit=1000&period="+i).json()
File "C:\Users\CarlosMiguel\Anaconda3\lib\site-packages\requests\models.py", line 892, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\CarlosMiguel\Anaconda3\lib\json\__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "C:\Users\CarlosMiguel\Anaconda3\lib\json\decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\CarlosMiguel\Anaconda3\lib\json\decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Currently It's written like this (everything that needs to be is imported):
try:
get_data = session.get(api_data_link+symbol+"?limit=1000&period="+i).json()
for candle in get_data:
#Do things with the requested data....
except:
print("JSONError")
print('Could not update data for')
print(symbol + ' ' + i)
Could it be that the API just doesn't return anything on some random call?
Because it seems to be completely random when it crashes with that error.
If it could be the API just not returning anything, how could I write it to call it again for a few times, before continuing on with the rest of the app?

json decod error when using POST in chalice

When I try to use app.current_request.json_body in chalice I get a decode error:
Traceback (most recent call last): File "/var/task/chalice/app.py",
line 659, in _get_view_function_response
response = view_function(**function_args) File "/var/task/app.py", line 34, in post_item
data = app.current_request.json_body File "/var/task/chalice/app.py", line 303, in json_body
self._json_body = json.loads(self.raw_body) File "/var/lang/lib/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s) File "/var/lang/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/var/lang/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char
0)
It doesn't matter how simple the data is. Example: {"Company":"ABC"} or {}.
As can be seen in the following code in the API Gateway all I try to do is return the data that has been sent so I don't think this is the problem:
#app.route('/test', methods=['POST'], content_types=['application/json'], cors=cors_config)
def post_item(data):
data = app.current_request.json_body
return data
Does anyone know what I might have done wrong?
You must remove the data from the parameters of the function.
That is used for pass url parameters.
#app.route('/test', methods=['POST'], content_types=['application/json'], cors=cors_config)
def post_item():
data = app.current_request.json_body
return data

Python 3.6: get JSON from aiohttp request

I have some application which uses aiohttp.
I sent POST request into approptiate endpoint, e.g.:
POST mysite.com/someendpoind/
with data similar to:
{"param1": "value1", "param2": "value2", ..., "paramn": None}
Then on backend side, I want to add some additional conditional into this request:
data = await request.json()
data["additional_conditional"] = True
But request.json() fails with an error:
[ERROR] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/aiohttp/web_protocol.py", line 422, in start
resp = yield from self._request_handler(request)
File "/usr/local/lib/python3.5/dist-packages/aiohttp/web.py", line 306, in _handle
resp = yield from handler(request)
File "/usr/local/lib/python3.5/dist-packages/aiohttp_session/__init__.py", line 129, in middleware
response = yield from handler(request)
File "/opt/bikeamp/auth/__init__.py", line 57, in wrapped
return (yield from f(request, user))
File "<my_module>.py", line 185, in <my_func>
data_json = await request.json()
File "/usr/local/lib/python3.5/dist-packages/aiohttp/web_request.py", line 469, in json
return loads(body)
File "/usr/lib/python3.5/json/__init__.py", line 319, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Then I decided to check somehow what is the content of my request:
await request.read()
b'field1=value1&field2=value2&field3=value3&field4=&field5=&field6='
So, I'm not sure, but the problem may be with empty parameters.
Also, I was trying to get this data via:
data = await request.post()
data["additional_condition"] = True
But this returns MultiDictProxy. Python can't pickle these objects.
Is there any known solutions?
I had the same issue, if post was something like {"email": "some#email.com"} check it with:
#router('/', methods=['POST', ])
async def post_request(request):
post = await request.post()
email = post.get('email') # because it's MultiDict
logging.warning(post) # see post details
logging.warning(email) # shows value "some#email.com"
json = await request.text() #
logging.warning(json) # shows json if it was ajax post request