While loading below json into json.loads(), I get error on "value" as it does not contain "".
jstring = '{"ABC": {value: "2787456", basevalue: "34453176"}}'
Keys need to be enclosed in double quotes:
>>> jstring='{"ABC":{"value":"2787456","basevalue":"34453176"}}'
>>> import json
>>> json.loads(jstring)
{u'ABC': {u'value': u'2787456', u'basevalue': u'34453176'}}
The problem is that value must also be enclosed in quote marks like this "value", in order to be parsed as json key.
jstring = '{"ABC": {"value": "2787456", "base value": "34453176"}}'
Related
i try to construct JSON with string that contains "\n" in it like this :
ver_str= 'Package ID: version_1234\nBuild\nnumber: 154\nBuilt\n'
proj_ver_str = 'Version_123'
comb = '{"r_content": {0}, "s_version": {1}}'.format(ver_str,proj_ver_str)
json_content = json.loads()
d =json.dumps(json_content )
getting this error:
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Dev/python/new_tester/simple_main.py", line 18, in <module>
comb = '{"r_content": {0}, "s_version": {1}}'.format(ver_str,proj_ver_str)
KeyError: '"r_content"'
The error arises not because of newlines in your values, but because of { and } characters in your format string other than the placeholders {0} and {1}. If you want to have an actual { or a } character in your string, double them.
Try replacing the line
comb = '{"r_content": {0}, "s_version": {1}}'.format(ver_str,proj_ver_str)
with
comb = '{{"r_content": {0}, "s_version": {1}}}'.format(ver_str,proj_ver_str)
However, this will give you a different error on the next line, loads() missing 1 required positional argument: 's'. This is because you presumably forgot to pass comb to json.loads().
Replacing json.loads() with json.loads(comb) gives you another error: json.decoder.JSONDecodeError: Expecting value: line 1 column 15 (char 14). This tells you that you've given json.loads malformed JSON to parse. If you print out the value of comb, you see the following:
{"r_content": Package ID: version_1234
Build
number: 154
Built
, "s_version": Version_123}
This isn't valid JSON, because the string values aren't surrounded by quotes. So a JSON parsing error is to be expected.
At this point, let's take a look at what your code is doing and what you seem to want it to do. It seems you want to construct a JSON string from your data, but your code puts together a JSON string from your data, parses it to a dict and then formats it back as a JSON string.
If you want to create a JSON string from your data, it's far simpler to create a dict with your values and use json.dumps on that:
d = json.dumps({"r_content": ver_str, "s_version": proj_ver_str})
So I have the following dictionaries that I get by parsing a text file
keys = ["scientific name", "common names", "colors]
values = ["somename1", ["name11", "name12"], ["color11", "color12"]]
keys = ["scientific name", "common names", "colors]
values = ["somename2", ["name21", "name22"], ["color21", "color22"]]
and so on. I am dumping the key value pairs using a dictionary to a json file using a for loop where I go through each key value pair one by one
for loop starts
d = dict(zip(keys, values))
with open("file.json", 'a') as j:
json.dump(d, j)
If I open the saved json file I see the contents as
{"scientific name": "somename1", "common names": ["name11", "name12"], "colors": ["color11", "color12"]}{"scientific name": "somename2", "common names": ["name21", "name22"], "colors": ["color21", "color22"]}
Is this the right way to do it?
The purpose is to query the common name or colors for a given scientific name. So then I do
with open("file.json", "r") as j:
data = json.load(j)
I get the error, json.decoder.JSONDecodeError: Extra data:
I think this is because I am not dumping the dictionaries in json in the for loop correctly. I have to insert some square brackets programatically. Just doing json.dump(d, j) won't suffice.
JSON may only have one root element. This root element can be [], {} or most other datatypes.
In your file, however, you get multiple root elements:
{...}{...}
This isn't valid JSON, and the error Extra data refers to the second {}, where valid JSON would end instead.
You can write multiple dicts to a JSON string, but you need to wrap them in an array:
[{...},{...}]
But now off to how I would fix your code. First, I rewrote what you posted, because your code was rather pseudo-code and didn't run directly.
import json
inputs = [(["scientific name", "common names", "colors"],
["somename1", ["name11", "name12"], ["color11", "color12"]]),
(["scientific name", "common names", "colors"],
["somename2", ["name21", "name22"], ["color21", "color22"]])]
for keys, values in inputs:
d = dict(zip(keys, values))
with open("file.json", 'a') as j:
json.dump(d, j)
with open("file.json", 'r') as j:
print(json.load(j))
As you correctly realized, this code failes with
json.decoder.JSONDecodeError: Extra data: line 1 column 105 (char 104)
The way I would write it, is:
import json
inputs = [(["scientific name", "common names", "colors"],
["somename1", ["name11", "name12"], ["color11", "color12"]]),
(["scientific name", "common names", "colors"],
["somename2", ["name21", "name22"], ["color21", "color22"]])]
jsonData = list()
for keys, values in inputs:
d = dict(zip(keys, values))
jsonData.append(d)
with open("file.json", 'w') as j:
json.dump(jsonData, j)
with open("file.json", 'r') as j:
print(json.load(j))
Also, for python's json library, it is important that you write the entire json file in one go, meaning with 'w' instead of 'a'.
I'm trying to create a multi-line String in Scala as below.
val errorReport: String =
"""
|{
|"errorName":"blah",
|"moreError":"blah2",
|"errorMessage":{
| "status": "bad",
| "message": "Unrecognized token 'noformatting': was expecting 'null', 'true', 'false' or NaN
at [Source: (ByteArrayInputStream); line: 1, column: 25]"
| }
|}
"""
.stripMargin
It's a nested JSON and it's not displaying properly when I print it. The message field inside errorMessage (which is the output of calling getMessage on an instance of a Throwable) is causing the issue because it looks like there is a newline right before
at [Source: ....
If I get rid of that line the JSON displays properly. Any ideas on how to properly format this are appreciated.
EDIT: The issue is with the newline character. So I think the question is more concisely - how to handle the newline within the triple quotes so that it's still recognized as a JSON?
EDIT 2: message is being set by a variable like so:
"message": "${ex.getMessage}"
where ex is a Throwable. An example of the contents of that getMessage call is provided above.
I assume that your question has nothing to do with JSON, and that you're simply asking how to create very wide strings without violating the horizontal 80-character limit in your Scala code. Fortunately, Scala's string literals have at least the following properties:
You can go from ordinary code to string-literal mode using quotes "..." and triple quotes """...""".
You can go from string-literal mode to ordinary code mode using ${...}
Free monoid over characters is reified as methods, that is, there is the + operation that concatenates string literals.
The whole construction can be made robust to whitespace and indentation using | and stripMargin.
All together, it allows you to write down arbitrary string literals without ever violating horizontal character limits, in a way that is robust w.r.t. indentation.
In this particular case, you want to make a line break in the ambient scala code without introducing a line break in your text. For this, you simply
exit the string-literal mode by closing """
insert concatenation operator + in code mode
make a line-break
indent however you want
re-enter the string-literal mode again by opening """
That is,
"""blah-""" +
"""blah"""
will create the string "blah-blah", without line break in the produced string.
Applied to your concrete problem:
val errorReport: String = (
"""{
| "errorName": "blah",
| "moreError": "blah2",
| "errorMessage": {
| "status": "bad",
| "message": "Unrecognized token 'noformatting'""" +
""": was expecting 'null', 'true', 'false' or NaN at """ +
"""[Source: (ByteArrayInputStream); line: 1, column: 25]"
| }
|}
"""
).stripMargin
Maybe a more readable option would be to construct the lengthy message separately from the neatly indented JSON, and then use string interpolation to combine the two components:
val errorReport: String = {
val msg =
"""Unrecognized token 'noformatting': """ +
"""was expecting 'null', 'true', 'false' or NaN at """ +
"""[Source: (ByteArrayInputStream); line: 1, column: 25]"""
s"""{
| "errorName": "blah",
| "moreError": "blah2",
| "errorMessage": {
| "status": "bad",
| "message": "${msg}"
| }
|}
"""
}.stripMargin
If the message itself contains line breaks
Since JSON does not allow multiline string literals, you have to do something else:
To remove line breaks, use .replaceAll("\\n", "") or rather .replaceAll("\\n", " ")
To encode line breaks with the escape sequence \n, use .replaceAll("\\n", "\\\\n") (yes... backslashes...)
I'm importing webservice tests from Excel and serialising them as YAML.
But taking advantage of YAML being a superset of JSON I'd like the request part of the test to be valid JSON, i.e. to have delimeters, quotes and commas.
This will allow us to cut and paste requests between the automated test suite and manual test tools (e.g. Postman.)
So here's how I'd like a test to look (simplified):
- properties:
METHOD: GET
TYPE: ADDRESS
Request URL: /addresses
testCaseId: TC2
request:
{
"unitTypeCode": "",
"unitNumber": "15",
"levelTypeCode": "L",
"roadNumber1": "810",
"roadName": "HAY",
"roadTypeCode": "ST",
"localityName": "PERTH",
"postcode": "6000",
"stateTerritoryCode": "WA"
}
In Python, my request object has a dict attribute called fields which is the part of the object to be serialised as JSON. This is what I tried:
import yaml
def request_presenter(dumper, request):
json_string = json.dumps(request.fields, indent=8)
return dumper.represent_str(json_string)
yaml.add_representer(Request, request_presenter)
test = Test(...including embedded request object)
serialised_test = yaml.dump(test)
I'm getting:
- properties:
METHOD: GET
TYPE: ADDRESS
Request URL: /addresses
testCaseId: TC2
request: "{
\"unitTypeCode\": \"\",\n
\"unitNumber\": \"15\",\n
\"levelTypeCode": \"L\",\n
\"roadNumber1\": \"810\",\n
\"roadName\": \"HAY\",\n
\"roadTypeCode\": \"ST\",\n
\"localityName\": \"PERTH\",\n
\"postcode\": \"6000\",\n
\"stateTerritoryCode\": \"WA\"\n
}"
...only worse because it's all on one line and has white space all over the place.
I tried using the | style for literal multi-line strings which helps with the line breaks and escaped quotes (it's more involved but this answer was helpful.) However, escaped or multiline, the result is still a string that will need to be parsed separately.
How can I stop PyYaml analysing the JSON block as a string and make it just accept a block of text as part of the emitted YAML? I'm guessing it's something to do with overriding the emitter but I could use some help. If possible I'd like to avoid post-processing the serialised test to achieve this.
Ok, so this was the solution I came up with. Generate the YAML with a placemarker ahead of time. The placemarker marks the place where the JSON should be inserted, and also defines the root-level indentation of the JSON block.
import os
import itertools
import json
def insert_json_in_yaml(pre_insert_yaml, key, obj_to_serialise):
marker = '%s: null' % key
marker_line = line_of_first_occurrence(pre_insert_yaml, marker)
marker_indent = string_indent(marker_line)
serialised = json.dumps(obj_to_serialise, indent=marker_indent + 4)
key_with_json = '%s: %s' % (key, serialised)
serialised_with_json = pre_insert_yaml.replace(marker, key_with_json)
return serialised_with_json
def line_of_first_occurrence(basestring, substring):
"""
return line number of first occurrence of substring
"""
lineno = lineno_of_first_occurrence(basestring, substring)
return basestring.split(os.linesep)[lineno]
def string_indent(s):
"""
return indentation of a string (no of spaces before a nonspace)
"""
spaces = ''.join(itertools.takewhile(lambda c: c == ' ', s))
return len(spaces)
def lineno_of_first_occurrence(basestring, substring):
"""
return line number of first occurrence of substring
"""
return basestring[:basestring.index(substring)].count(os.linesep)
embedded_object = {
"unitTypeCode": "",
"unitNumber": "15",
"levelTypeCode": "L",
"roadNumber1": "810",
"roadName": "HAY",
"roadTypeCode": "ST",
"localityName": "PERTH",
"postcode": "6000",
"stateTerritoryCode": "WA"
}
yaml_string = """
---
- properties:
METHOD: GET
TYPE: ADDRESS
Request URL: /addresses
testCaseId: TC2
request: null
after_request: another value
"""
>>> print(insert_json_in_yaml(yaml_string, 'request', embedded_object))
- properties:
METHOD: GET
TYPE: ADDRESS
Request URL: /addresses
testCaseId: TC2
request: {
"unitTypeCode": "",
"unitNumber": "15",
"levelTypeCode": "L",
"roadNumber1": "810",
"roadName": "HAY",
"roadTypeCode": "ST",
"localityName": "PERTH",
"postcode": "6000",
"stateTerritoryCode": "WA"
}
after_request: another value
I have this json data that I post to Djangp
[[LAT2:1.3178775, LON1:103.7608174, LON2:103.7733836, LAT1:1.3104325, YPIXELS:378, XPIXELS:400, MINZ:40, XKNOTS:26, YKNOTS:24, MAXZ:80], [node:51, z:63.462589, y:1.312762, x:103.766148]]
Getting error: Expecting value: line 1 column 3 (char 2)
My codes:
jsonStr = request.body.decode(encoding='UTF-8')
jsonObj = json.loads(jsonStr)
Your string isn't valid json. key:value pairs need to be encompassed in curly brackets, and keys must be quoted. For example: {"key" : "value"}
At minimum you're going to have to transform your string into this format:
[{"LAT2":1.3178775, "LON1":103.7608174, "LON2":103.7733836, "LAT1":1.3104325, "YPIXELS":378, "XPIXELS":400, "MINZ":40, "XKNOTS":26, "YKNOTS":24, "MAXZ":80}, {"node":51, "z":63.462589, "y":1.312762, "x":103.766148}]'
Here's a way to do it without regular expressions, if you know the keys in the string will be consistent:
# Bracket the {key : value} structures:
input = '[' + input[1:-1].replace('[', '{').replace(']', '}') + ']'
# Wrap the keys in double quotes:
keys = ('LAT2', 'LON1', 'LON2', 'LAT1', 'YPIXELS', 'XPIXELS',
'XKNOTS', 'YKNOTS', 'MINZ', 'MAXZ', 'node', 'z', 'y', 'x')
for key in keys:
input = input.replace(key, '"' + key + '"')