App randomly crashes with a JSON error - json

I've posted first the error message with full traceback and the code as well.
The code that is below is within a loop, that will run multiple times changing the variables within the session.get() link for the API data call.
From time to time I will get an error that the json.decoder.JSONDecodeError is expecting value... (full error below)
I can't figure out why.
Traceback (most recent call last):
File "C:\Users\CarlosMiguel\Desktop\experiments\data_collector\data_collector.py", line 100, in <module>
get_data = session.get(api_data_link+symbol+"?limit=1000&period="+i).json()
File "C:\Users\CarlosMiguel\Anaconda3\lib\site-packages\requests\models.py", line 892, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\CarlosMiguel\Anaconda3\lib\json\__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "C:\Users\CarlosMiguel\Anaconda3\lib\json\decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\CarlosMiguel\Anaconda3\lib\json\decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Currently It's written like this (everything that needs to be is imported):
try:
get_data = session.get(api_data_link+symbol+"?limit=1000&period="+i).json()
for candle in get_data:
#Do things with the requested data....
except:
print("JSONError")
print('Could not update data for')
print(symbol + ' ' + i)
Could it be that the API just doesn't return anything on some random call?
Because it seems to be completely random when it crashes with that error.
If it could be the API just not returning anything, how could I write it to call it again for a few times, before continuing on with the rest of the app?

Related

Telegram bot with pythonanywhere. JSON issue

I'm trying to deploy Telegram bot on pythonanywhere. Here is my code. All was working until I updated repository. Nothing was change in this piece of code. But it happed not working now. The get_message function not even calls. May be it is some problems with JSON or so, i don't understand exactly. How to get rid of this error?
URL = 'name.pythonanywhere.com'
app = Flask(__name__)
sslify = SSLify(app)
bot.remove_webhook()
bot.set_webhook(url=URL)
bot.send_message(userID, 'helo')
#app.route('/', methods=['POST', 'GET'])
def get_message():
update = types.Update.de_json(request.stream.read().decode('utf-8'))
bot.process_new_updates([update])
return 'ok', 200
2023-01-15 22:15:05,776: Exception on / [GET]
Traceback (most recent call last):
File "/home/kentus/.local/lib/python3.9/site-packages/flask/app.py", line 2077, in wsgi_app
response = self.full_dispatch_request()
File "/home/kentus/.local/lib/python3.9/site-packages/flask/app.py", line 1526, in full_dispatch_request
return self.finalize_request(rv)
File "/home/kentus/.local/lib/python3.9/site-packages/flask/app.py", line 1545, in finalize_request
response = self.make_response(rv)
File "/home/kentus/.local/lib/python3.9/site-packages/flask/app.py", line 1701, in make_response
raise TypeError(
TypeError: The view function for 'get_message' did not return a valid response. The function either returned None or ended without a return statement.
2023-01-15 22:52:17,579: Exception on / [GET]
Traceback (most recent call last):
File "/home/kentus/.local/lib/python3.9/site-packages/flask/app.py", line 2077, in wsgi_app
response = self.full_dispatch_request()
File "/home/kentus/.local/lib/python3.9/site-packages/flask/app.py", line 1525, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/kentus/.local/lib/python3.9/site-packages/flask/app.py", line 1523, in full_dispatch_request
rv = self.dispatch_request()
File "/home/kentus/.local/lib/python3.9/site-packages/flask/app.py", line 1509, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "/home/kentus/english_bot/app/main.py", line 177, in get_message
update = types.Update.de_json(request.stream.read().decode('utf-8'))
File "/home/kentus/.local/lib/python3.9/site-packages/telebot/types.py", line 96, in de_json
obj = cls.check_json(json_string, dict_copy=False)
File "/home/kentus/.local/lib/python3.9/site-packages/telebot/types.py", line 80, in check_json
return json.loads(json_type)
File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I tried to manipulate the webhook but it ok with it. I guess. getWebhookInfo method shows correct url.
i tried to run locally flask app with public URL using ngrok to debug it locally, but i had errors with setting webhook on ngrok URL.
I even asked ChatGPT and he didn't help.

How can I fix this error: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I am trying to create an REST API and a part of the project requires working with JSON files.
I am trying to open a JSON file, to check whether certain content exists in the file. If it doesn't, it will be added. For that reason, I am trying to load the JSON file in order to use the dictionary methods. When I open the file and try to use the json.loads() function, I get an error. I am attaching the part of the code that doesn't work.
file = open(self.storage_file_name, 'r')
if file_is_empty:
content_to_write = json.dumps(self.states_population_json, indent=4)
file.write(content_to_write)
else:
current_date = str(date.today())
print(file.read())
json_content = json.loads(file.read()) # THIS IS WHERE THE ISSUE APPEARS <<<
if current_date in self.states_population_json["features"].keys():
pass
else:
json_content["features"][current_date] = self.states_population_json["features"][current_date]
Here's the error as well:
Traceback (most recent call last):
File "C:\Users\denis\AppData\Local\Programs\Python\Python39-32\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I tried using json.load() instead of json.loads(), but then I get the following error:
Traceback (most recent call last):
File "C:\Users\denis\Desktop\Programming\ESRI_Internship\test_file.py", line 89, in <module>
data.save_data_json()
File "C:\Users\denis\Desktop\Programming\ESRI_Internship\test_file.py", line 79, in save_data_json
json_content = json.load(file)
File "C:\Users\denis\AppData\Local\Programs\Python\Python39-32\lib\json\__init__.py", line 293, in load
return loads(fp.read(),
File "C:\Users\denis\AppData\Local\Programs\Python\Python39-32\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\denis\AppData\Local\Programs\Python\Python39-32\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\denis\AppData\Local\Programs\Python\Python39-32\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Process finished with exit code 1
I made sure that the file isn't empty.
Thanks in advance!
EDIT:
Here is the content of the file (shortened a bit):
{
"features": {
"2022-01-06": [
{
"STATE_NAME": "Alabama",
"POPULATION": 5028316
},
{
"STATE_NAME": "North Carolina",
"POPULATION": 10736879
},
{
"STATE_NAME": "North Dakota",
"POPULATION": 321419
}
]
}
}
The line print(file.read()) moves the file cursor for reading to the end of the file (the command reads the whole file). A subsequent file.read() thus will not be able to read anything. Just remove the print statement, or if needed, move the file cursor back to the beginning of the file by executing file.seek(0) before the json.loads() statement.

Json loads fails on mqtt message

I'm trying to parse the messages sent on an MQTT topic as JSONs, but when i try to call json.loads() on the decoded string, the code goes into exception:
File "/usr/lib/python3.7/json/decoder.py", line 340, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 153 (char 152)
The code is the following:
def on_message_mqtt(self, client, userdata, message):
m_decode = message.payload.decode('utf-8')
logger.info(m_decode)
m_in = json.loads(m_decode)
logger.info(m_in)
While the payload of the messages has the following format:
{"Timestamp":"2021-05-24-13:27:13.450","AI":[1204,7,0,4,18,2,1176,802,11,0,381,2496,0,0,810,1282],"DI":[false,false,false,false,false,false,false,false]}
I tried to put the text into a string and loads() is able to parse it, so I bet that the problem is on the initial string decoding.
Here you can see a print of the decoded string, and then the error on the call to json.loads():
2021-05-24 16:31:47 - INFO - {"Timestamp":"2021-05-24-14:31:44.790","AI":[912,9,0,2,16,2,886,605,11,0,321,1924,1,963,620,9],"DI":[false,false,false,false,false,false,false,false]}
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "router.py", line 136, in run
self.mqtt.loop_forever()
File "/opt/control/env/lib/python3.7/site-packages/paho/mqtt/client.py", line 1779, in loop_forever
rc = self.loop(timeout, max_packets)
File "/opt/control/env/lib/python3.7/site-packages/paho/mqtt/client.py", line 1181, in loop
rc = self.loop_read(max_packets)
File "/opt/control/env/lib/python3.7/site-packages/paho/mqtt/client.py", line 1572, in loop_read
rc = self._packet_read()
File "/opt/control/env/lib/python3.7/site-packages/paho/mqtt/client.py", line 2310, in _packet_read
rc = self._packet_handle()
File "/opt/control/env/lib/python3.7/site-packages/paho/mqtt/client.py", line 2940, in _packet_handle
return self._handle_pubrel()
File "/opt/control/env/lib/python3.7/site-packages/paho/mqtt/client.py", line 3246, in _handle_pubrel
self._handle_on_message(message)
File "/opt/control/env/lib/python3.7/site-packages/paho/mqtt/client.py", line 3444, in _handle_on_message
self.on_message(self, self._userdata, message)
File "router.py", line 87, in on_message_mqtt
logger.info(json.loads(m_decode), exc_info=True)
File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.7/json/decoder.py", line 340, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 151 (char 150)
Judging from the error, character 152 is the square bracket closing the arroy of the value for the key of "DI" ] which would imply incorrect parsing using the payload.decode function
Can you check if this works
def on_message_mqtt(self, client, userdata, message):
m_in = json.loads(message.decode("utf-8"))
logger.info(m_in)

JSONDecodeError: Expecting value: line 1 column 1 error in AWS Lambda

I am trying to write a aws lambda function which will push the SQS queue output in a s3 bucket.
But the lambda function is failing to push the message , the cloudwatch log is showing
JSONDecodeError: Expecting value: line 1 column 1
i am posting the lambda function which i am using
import json
import boto3
def lambda_handler(event, context):
s3 = boto3.client("s3")
data = json.loads(event["Records"][0]["body"]) --getting error in this line
print(data)
s3.put_object(Bucket="sqsmybucket",key="data.json", Body=json.dumps(data))
#print(event)
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
The cloud watch log is showing
2020-05-30T23:51:45.276+05:30
[ERROR] JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 6, in lambda_handler
data = json.loads(event["Records"][0]["body"])
File "/var/lang/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/var/lang/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/var/lang/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
I have formatted the message and saved it to the cloud below is the link
[formatted JSON code][1]
formatted JSON
Please help , thanks in advance
Your event["Records"][0]["body"] is a plain string, not json:
"body": "A difficult message."
Therefore, json.loads(event["Records"][0]["body"]) is equivalent to json.loads("A difficult message.") which obviously fails.
To get body's value you can do the following instead:
data = event["Records"][0]["body"]
However, since later you have the following statment:
Body=json.dumps(data)
The Body will be:
Body='"A difficult message."'
which may or may not be what you desire.

json decod error when using POST in chalice

When I try to use app.current_request.json_body in chalice I get a decode error:
Traceback (most recent call last): File "/var/task/chalice/app.py",
line 659, in _get_view_function_response
response = view_function(**function_args) File "/var/task/app.py", line 34, in post_item
data = app.current_request.json_body File "/var/task/chalice/app.py", line 303, in json_body
self._json_body = json.loads(self.raw_body) File "/var/lang/lib/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s) File "/var/lang/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/var/lang/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char
0)
It doesn't matter how simple the data is. Example: {"Company":"ABC"} or {}.
As can be seen in the following code in the API Gateway all I try to do is return the data that has been sent so I don't think this is the problem:
#app.route('/test', methods=['POST'], content_types=['application/json'], cors=cors_config)
def post_item(data):
data = app.current_request.json_body
return data
Does anyone know what I might have done wrong?
You must remove the data from the parameters of the function.
That is used for pass url parameters.
#app.route('/test', methods=['POST'], content_types=['application/json'], cors=cors_config)
def post_item():
data = app.current_request.json_body
return data