I have following json file:
{'a': {'$gt': datetime.datetime(2020, 1, 1, 0, 0)}}
I converted it into string and replaces ' --> "":
>>> x
'{"a": {"$gt": datetime.datetime(2020, 1, 1, 0, 0)}}'
Getting error in loading the json:
>>> json.loads(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python36\lib\json\__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "C:\Python36\lib\json\decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python36\lib\json\decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 15 (char 14)
The issue has got resolved. The issue arising from the fact : “datetime.datetime not JSON serialize”
from json import dumps
from datetime import date, datetime
def json_serial(obj):
"""JSON serializer for objects not serializable by default json code"""
if isinstance(obj, (datetime, date)):
return obj.isoformat()
raise TypeError ("Type %s not serializable" % type(obj))
After importing this function:
>>> x
{'a': {'$gt': datetime.datetime(2020, 1, 1, 0, 0)}}
>>> x=dumps(x,default=json_serial)
>>> x
'{"a": {"$gt": "2020-01-01T00:00:00"}}'
>>>
>>>
>>> json.loads(x)
{'a': {'$gt': '2020-01-01T00:00:00'}}
>>>
Related
The issue:
>>> import json
>>> json_str='{"message": "John said "come here!""}'
>>> json.loads(json_str)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/ram/.pyenv/versions/3.9.6/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/Users/ram/.pyenv/versions/3.9.6/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/ram/.pyenv/versions/3.9.6/lib/python3.9/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 25 (char 24)
Then I attempted to escape the double-quotes. Like so:
>>> json_str='{"message": "John said \"come here!\""}'
>>> json.loads(json_str)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/ram/.pyenv/versions/3.9.6/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/Users/ram/.pyenv/versions/3.9.6/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/ram/.pyenv/versions/3.9.6/lib/python3.9/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 25 (char 24)
The following works though:
>>> json_str='{"message": "John said \'come here!\'"}'
>>> json.loads(json_str)
{'message': "John said 'come here!'"}
What am I doing wrong? Is it possible to flag the json library to be more forgiving (helps me leave json parsing to the json library)?
PS: Following the answer here, I attempted the following:
>>> data=json.dumps('{"message":"John said "hello!""}')
>>> print(data)
"{\"message\":\"John said \"hello!\"\"}"
>>> dict = json.loads(data)
>>> print(dict)
{"message":"John said "hello!""}
>>> print(dict['message'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: string indices must be integers
>>> print(dict[0])
{
The last json.loads is returning a string instead of dict. But the correctly escaped quotes in data did sound promising.
You can simply use either a raw string or double backslashes so that you both escape the backslashes in python as well as in the json string so that json.loads can see the backslashes.
>>> json.loads(r'{"message": "John said \"come here!\""}')
{'message': 'John said "come here!"'}
>>> json.loads('{"message": "John said \\"come here!\\""}')
{'message': 'John said "come here!"'}
Let say I have a simple dictionary as below:
a = {'animal' : 'fish', 'fruit' : 'apple','vehicle' : 'car'}
I wish to convert a into JSON and write to a file. If I used json.dumps() as below:
b = jason.dumps(a)
with open('output.json', 'w') as f:
f.write(pprint.pformat(b))
The content of the JSON file output.json became:
'{"animal": "fish", "fruit": "apple", "vehicle": "car"}'
What I wish to actually have (in the file) is without quote as below:
{"animal": "fish", "fruit": "apple", "vehicle": "car"}
What is the right way to code it so that I can get the desired output?
Thanks in advance.
[EDIT]: The reason why I need the desired output in file is because I received the error below when I loaded the file.
>>> with open("output.json") as f:
... data = json.load(f)
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/Users/louis.kok/Analyses/devel/anaconda3/lib/python3.7/json/__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/Users/louis.kok/Analyses/devel/anaconda3/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/Users/louis.kok/Analyses/devel/anaconda3/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/louis.kok/Analyses/devel/anaconda3/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Import json library
import json
Save your file properly using json.dump
with open('output.json', 'w') as f:
json.dump(a, f)
Read it using json.load
with open("output.json") as f:
data = json.load(f)
Try this:
with open('task.json', 'w') as f:
s = json.dumps(mydict, indent=2)
f.write('%s\n' % (s))
I have a json object file and at the beginning of this file a header of text, date, time, IP address.
I've tried f.readlines()[5:] to no avail.
I've tried next(f)
I wish to skip 5 or 6 lines of text and go directly into the json data.
Here is an example.
import jsonlines
import json
data_file = input("Enter a file to parse: ")
with jsonlines.open(data_file) as file:
for obj in file:
try:
jsonparse = json.loads(obj)
except Exception as e:
pass
print(obj)
Error:
jsonlines.jsonlines.InvalidLineError: line contains invalid json: Expecting value: line 1 column 1 (char 0) (line 1)
Top of json file:
Start: 07/02/2019 14:59:40.686
Connected To:
192.168.11.203
Here is the full long error:
Enter a file to parse: Play.raw
Traceback (most recent call last):
File "C:\Users\sdickey\AppData\Local\Programs\Python\Python37\lib\site-packages\jsonlines\jsonlines.py", line 159, in read
value = self._loads(line)
File "C:\Users\sdickey\AppData\Local\Programs\Python\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\sdickey\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\sdickey\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:/Users/sdickey/PycharmProjects/Python Testing/Actall Data/testing.py", line 6, in <module>
for obj in file:
File "C:\Users\sdickey\AppData\Local\Programs\Python\Python37\lib\site-packages\jsonlines\jsonlines.py", line 204, in iter
skip_empty=skip_empty)
File "C:\Users\sdickey\AppData\Local\Programs\Python\Python37\lib\site-packages\jsonlines\jsonlines.py", line 164, in read
six.raise_from(exc, orig_exc)
File "<string>", line 3, in raise_from
jsonlines.jsonlines.InvalidLineError: line contains invalid json: Expecting value: line 1 column 1 (char 0) (line 1)
Process finished with exit code 1
It is always preferable not to read the whole file in memory but one line at a time.
Assuming your input file contains:
first line
second line
third line
fourth line
fifth line
{ "k1": "val1", "k2": "val2" }
{ "k3": "val3", "k4": "val4" }
if you just want to skip 5 lines, you could do it brutally as:
import json
with open("test.txt") as f:
for _ in range(5):
next(f)
for line in f:
obj = json.loads(line)
print(obj)
or using enumerate:
import json
with open("test.txt") as f:
for i, line in enumerate(f):
if i<5:
continue
obj = json.loads(line)
print(obj)
or use itertools' dropwhile:
import itertools as it
import json
with open("test.txt") as f:
for i, line in it.dropwhile(lambda i: i[0]<5, enumerate(f)):
obj = json.loads(line)
print(obj)
I think you are trying to convert to json line by line. You need to join all lines by \n igoring the first 5 lines and then load it:
import json
with open("test.txt") as f:
json_obj = "\n".join(f.readlines()[5:])
jsonparse = json.loads(json_obj)
print(jsonparse)
Pulled details of server from internal URL's API using Python with SOAP+XML
Code:
import requests
import xml.etree.ElementTree as ET
headers = {'content-type': 'application/soap+xml'}
body = """<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<SOAP-ENV:Header>
<m:Username xmlns:m="http://www.ab.com">test</m:Username>
<m:Password xmlns:m="http://www.ab.com">testpass</m:Password>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<m:GetCI xmlns:m="http://www.ab.com">
<CIType xsi:type="xsd:string">System</CIType>
<CIName xsi:type="xsd:string">server.example.nl</CIName>
<CIID></CIID>
<AttrFilter xsi:type="xsd:string">PrimaryName+ArpaDomain+SystemStatus+Environment+ServiceLevel+IsVirtual+Coverage</AttrFilter>
<SubObjFilter xsi:type="xsd:string"></SubObjFilter>
</m:GetCI>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>"""
r = requests.post("URL of API", data=body, headers=headers, verify=False)
print(r.text)
SOAP XML response which i got:
'<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Header><Username xmlns="http://www.ab.com">test</Username><Password xmlns="http://www.hp.com">testpass</Password></soap:Header><soap:Body><ns1:GetCIResponse xmlns:ns1="http://www.ab.com"><returnWord><![CDATA[{"Attributes":{"PrimaryName":"server","ArpaDomain":"example.nl","SystemStatus":"obsolete","Environment":"Development","ServiceLevel":"standard","
IsVirtual":"no","Coverage":"24x7 (00:00-24:00 Mon-Sun)"},"DataCenter":{"DCID":"1041166","SourceID":null,"SourceTool":null},{"InstanceID":"159364108","SolutionName":"oracle engine","SolutionCategory":"engine","InstanceName":null,"InstanceStatus":"deinstalled","InstanceEnvironment":"Test","InstanceImpact":"3 - Multiple Users","InstanceServiceLevel":"standard","
InstanceAvailability":"99.98","InstanceTimezone":"(GMT+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna","InstanceCoverage":"11x5 (07:00-18:00 Mon-Fri)","InstanceVersion":null,"InstanceDescription":"Acceptance - Pe
oplesoft Financials Shadow","InstanceAssignmentGroup":"E-INCSSP-APAC-MDR-DBA-EHM","SourceID":null,"SourceTool":null}],"BusinessID":"10292","Owner":"1","User":"1","ACTION":null}],"MaintContracts":[{"ContractName":"123456","ContractStart":null,"ContractEnd":null,"CRCName":null,"SystemHandle":null,"ContractInfo":null,"ACTION":nul
l},{"ContractName":"RedHat Support","ContractStart":null,"ContractEnd":null,"CRCName":null,"SystemHandle":null,"ContractInfo":null,"ACTION":null}],
"Params":[{"ParCd":"AVBM","SrsValue":"Option 3","ACTION":null},{"ParCd":"BILLABLE","SrsValue":"Yes","ACTION":null},{"ParCd":"BT_SERVICE_LEVEL","SrsValue":"Bronze","ACTION":null},{"ParCd":"CPU_TYPE","SrsValue":"Intel(R) Xeon(R) CPU E5530 # 2.40GHz","ACTION":null},{"ParCd":"DRP_PRIORITY","SrsValue":"DR4","ACTION":null},{"ParCd":"RIM_LOAD","SrsValue":"06 JUL 2017 09:41:00","ACTION":null}],
"ABSA":[{"MID":"67260252","MeshID":"4","MeshShortName":"APAC_sdn","MeshName":"AB-SA APAC SDN"}]}]]></returnWord></ns1:GetCIResponse></soap:Body></soap:Envelope>'
Converting string output to Dictionary:
>>> type(r.text)
<class 'str'>
>>> data=json.loads(r.text)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\dupakunt\AppData\Local\Programs\Python\Python36-32\lib\json\__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "C:\Users\dupakunt\AppData\Local\Programs\Python\Python36-32\lib\json\decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\dupakunt\AppData\Local\Programs\Python\Python36-32\lib\json\decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
How to Convert this output to dictionary and extract only "attributes".
import re
root = ET.fromstring(r.text)
for child in root.iter('returnWord'):
text=child.text
l=re.split("[{}]+", text)
details=l[2]
I am receiving this error. I am using Python 3.5.0.
Here is my code:
import json
import requests
from tqdm import tqdm
import os.path
r = requests.get("http://prod.ivtr-od.tpg.ch/v1/GetStops.json?key=d95be980-0830-11e5-a039-0002a5d5c51b")
path = "/Users/me/Desktop/JSON/"
pbar = tqdm(r.json()["stops"])
for x in pbar:
tree = {}
fileMON = open(path + x["stopCode"] + ".json", "r", encoding='utf8')
print(fileMON)
if "MON" in json.loads(fileMON()):
tree["MON"] = json.loads(fileMON())["MON"]
and this is the output :
<_io.TextIOWrapper name='/Users/me/Desktop/JSON/31DCdeparts.json' mode='r' encoding='utf8'>
Traceback (most recent call last):
File "json.py", line 14, in <module>
tree["MON"] = json.loads(fileMON())["MON"]
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/__init__.py", line 319, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
My JSON file is in the UTF8 encoding. Someone can help me ?