Groovy JsonSlurper JSON value with escaped quotation marks - json

I'm developing a small tool written in Groovy which parses JSON strings in emails. Some of these JSON strings have JSON values that contain escaped quotes.
For example:
{
"hello": "world with \"quotation marks\""
}
To parse these strings, I am using Groovy's JsonSlurper. The following code demonstrates my problem:
import groovy.json.JsonException
import groovy.json.JsonSlurper
try {
print new JsonSlurper().parseText('''
{
"hello": "world with \"quotation marks\""
}
''')
} catch (JsonException | IllegalArgumentException e) {
print e
}
See https://groovyconsole.appspot.com/script/6193189027315712 for a live demo.
When executing this code, the following exception is thrown:
groovy.json.JsonException: expecting '}' or ',' but got current char 'q' with an int value of 113
The current character read is 'q' with an int value of 113
expecting '}' or ',' but got current char 'q' with an int value of 113
line number 3
index number 35
"hello": "world with "quotation marks""
............................^
Thus, the escaping of the quotation mark is ignored by JsonSlurper. Unfortunately, I have no control over the input, i.e. the JSON strings. Therefore, I have to find a way to parse such a JSON string into a map or any other appropriate data structure.

The string has not been escaped properly in the json. The text data should be like:
'''
{
"hello": "world with \\\"quotation marks\\\""
}
'''
The string that you are getting indicates that the mail body contains the json in format:
{
"hello": "world with "quotation marks""
}
while it should be like
{
"hello": "world with \"quotation marks\""
}
If earlier is the case then you can't parse the invalid json as there is no way for code to identify about the escaped data.

Related

How to seperate two json files by string?

I am trying to parse a JSON from a website in order to extract some data. Somehow the JSON is "combined", meaning that the dataset for one product is described and without separating the next, the JSON simply continues. This throws an error in json.load (python3.8).
Here is the part in question:
"name": "30 stk"
}
}
} {
"#type":"Product"
jsonlint outputs the following error:
Error: Parse error on line 126:
...30 stk" } } } { "#type": "Produc
--------------------^
Expecting 'EOF', '}', ',', ']', got '{'
Full JSON:
{"#context":"https://schema.org","#type":"Product","#id":"/antistax-extra-venentabletten-bei-venenleiden-60stk-pzn-00002335","aggregateRating":{"#type":"AggregateRating","ratingValue":"4,8","reviewCount":"26"},"description":"AntistaxextraVenentablettenbeiVenenleiden60stkkaufenbeiderOnlineApothekeapo-discounter.Medikamente,Nahrungsergänzungenuvm.erhaltenSieinunsererVersandapothekezugünstigenPreisen.","name":"AntistaxextraVenentablettenbeiVenenleiden(60stk)","image":"https://www.apodiscounter.de/images/product_images/info_images/00002335_4.jpg","sku":"00002335","mpn":"00002335","productID":"00002335","category":"Venenerkrankung","brand":{"#type":"Organization","name":"Antistax"},"offers":{"#type":"Offer","availability":"https://schema.org/InStock","url":"https://www.apodiscounter.de/antistax-extra-venentabletten-bei-venenleiden-60stk-pzn-00002335","price":"27.49","priceValidUntil":"2021-01-26","priceCurrency":"EUR","category":"Filmtabletten","eligibleQuantity":"60stk","itemCondition":"NewCondition","seller":{"#type":"Organization","name":"apo-discounter.de"}},"issimilarto":[{"#type":"Product","mpn":"00002312","name":"AntistaxextraVenentablettenbeiVenenleiden","url":"https://www.apodiscounter.de/antistax-extra-venentabletten-bei-venenleiden-30stk-pzn-00002312","description":"AntistaxextraVenentablettenbeiVenenleiden30stkkaufenbeiderOnlineApothekeapo-discounter.Medikamente,Nahrungsergänzungenuvm.erhaltenSieinunsererVersandapothekezugünstigenPreisen.","image":"https://www.apodiscounter.de/images/product_images/info_images/00002312.jpg","sku":"00002312","brand":{"#type":"Organization","name":"Antistax"},"aggregateRating":{"#type":"AggregateRating","ratingValue":"5,0","reviewCount":"13"},"review":[{"#type":"Review","author":"ErnaD.","datePublished":"30.10.2020","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"DomenicaZ.","datePublished":"04.09.2020","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"RitaS.","datePublished":"11.07.2020","description":"kaumwirksam","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"VerifizierterKunde","datePublished":"10.04.2020","description":"Binnichtganzüberzeugt","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"VerifizierterKunde","datePublished":"06.03.2020","description":"BinpositivüberraschtvondemProdukt","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}}],"offers":{"#type":"Offer","availability":"https://schema.org/InStock","url":"https://www.apodiscounter.de/antistax-extra-venentabletten-bei-venenleiden-30stk-pzn-00002312","price":"15.99","priceValidUntil":"2021-01-26","priceCurrency":"EUR","eligibleQuantity":{"#type":"QuantitativeValue","name":"30stk"}}}{"#type":"Product","mpn":"05954715","name":"AntistaxextraVenentablettenbeiVenenschwäche","url":"https://www.apodiscounter.de/antistax-extra-venentabletten-bei-venenschwaeche-90stk-pzn-05954715","description":"AntistaxextraVenentablettenbeiVenenschwäche90stkkaufenbeiderOnlineApothekeapo-discounter.Medikamente,Nahrungsergänzungenuvm.erhaltenSieinunsererVersandapothekezugünstigenPreisen.","image":"https://www.apodiscounter.de/images/product_images/info_images/05954715.jpg","sku":"05954715","brand":{"#type":"Organization","name":"Antistax"},"aggregateRating":{"#type":"AggregateRating","ratingValue":"4,8","reviewCount":"167"},"review":[{"#type":"Review","author":"VerifizierterKunde","datePublished":"05.12.2020","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"RoswithaT.","datePublished":"23.11.2020","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"RalfZ.","datePublished":"22.11.2020","description":"Gutundpreiswert!","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"ReinhardS.","datePublished":"22.11.2020","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"VerifizierterKunde","datePublished":"08.11.2020","description":"WennmanProblememitVenenhatundlangeaufdenBeinenist,hilftesdenTagbesserzumeistern...","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}}],"offers":{"#type":"Offer","availability":"https://schema.org/InStock","url":"https://www.apodiscounter.de/antistax-extra-venentabletten-bei-venenschwaeche-90stk-pzn-05954715","price":"33.49","priceValidUntil":"2021-01-26","priceCurrency":"EUR","eligibleQuantity":{"#type":"QuantitativeValue","name":"90stk"}}},{"#type":"Product","mpn":"16156023","name":"AntistaxextraVenentablettenbeiVenenleiden&Venenschwäche","url":"https://www.apodiscounter.de/antistax-extra-venentabletten-bei-venenleiden-venenschwaeche-180stk-pzn-16156023","description":"AntistaxextraVenentablettenbeiVenenleiden&Venenschwäche180stkkaufenbeiderOnlineApothekeapo-discounter.Medikamente,Nahrungsergänzungenuvm.erhaltenSieinunsererVersandapothekezugünstigenPreisen.","image":"https://www.apodiscounter.de/images/product_images/info_images/16156023.jpg","sku":"16156023","aggregateRating":{"#type":"AggregateRating","ratingValue":"5,0","reviewCount":"3"},"review":[{"#type":"Review","author":"IrinaL.","datePublished":"20.01.2021","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"AlfredS.","datePublished":"20.01.2021","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"VerifizierterKunde","datePublished":"11.01.2021","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}}],"offers":{"#type":"Offer","availability":"https://schema.org/InStock","url":"https://www.apodiscounter.de/antistax-extra-venentabletten-bei-venenleiden-venenschwaeche-180stk-pzn-16156023","price":"56.49","priceValidUntil":"2021-01-26","priceCurrency":"EUR","eligibleQuantity":{"#type":"QuantitativeValue","name":"180stk"}}}],"review":[{"#type":"Review","author":"KlausS.","datePublished":"11.12.2020","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"4","worstRating":"1"}},{"#type":"Review","author":"OttiZ.","datePublished":"19.10.2020","description":"Zufrieden","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"MonikaM.","datePublished":"03.09.2020","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"RitaP.","datePublished":"14.08.2020","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"5","worstRating":"1"}},{"#type":"Review","author":"VerifizierterKunde","datePublished":"24.07.2020","description":"","name":"","reviewRating":{"#type":"Rating","bestRating":"5","ratingValue":"3","worstRating":"1"}}]}
How can I separate those two or is there a function for this? I tried .split('} {') but this did not work.
It seems you have a JSON array containing two JSON objects not separated by a comma.
You should just replace } { with }, {.

How to handle '\' in json date with python json.loads

I want to read texts from a json file and then convert it to a python dict object, but meet some problems when the backslash is in the json data.
what my json file is like:
{
"foo": "+|\\*|/",
}
I want to get a dict like this:
{
"foo": "+|\*|/",
}
but I get this actually:
{
"foo": "+|\\*|/",
}
If I change the json file like this:
{
"foo": "+|\*|/"
}
or
{
"foo": "+|\\\*|/",
}
then I will get an error.
The python versoin is 3.8.0
The problem may be not in the loading of the JSON file, but in displaying its content as Python object (since repr() is used under the hood, and returns the representation with single quotes and escaped characters).
For example, my file (named 1.json) is
{
"foo": "+|\\*|/"
}
I'm using the following code to load it in Python 3.8.0:
import json
a = json.loads(open("1.json").read())
Now print(a) prints {'foo': '+|\\*|/'}, while print(a["foo"]) will print the correct result: +|\*|/.
So, please check that your displayed part is not using repr().

Add escape character for ctrl-char in a json string

I am trying to json decode a string but I am getting an error as:
"message":"error while converting json to class","level":"ERROR","level_value":40000,"stack_trace":"com.fasterxml.jackson.databind.JsonMappingException: Illegal unquoted character ((CTRL-CHAR, code 31)): has to be escaped using backslash to be included in string value\n at [Source: { \"key\": \"abc No.\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F\u001F 106,xyz}
I am suspecting that there might be a ctrl-char character(/n) in the json. I want to know how add the extra escape character wherever this ctrl-char might be present, so that my code doesn't break while decoding the json.
Json Decode code:
public static Object jsonDecode(String s, Class c) {
Object u = null;
if(s!=null) {
try {
u = objectMapper.reader(c)
.readValue(s);
} catch (Exception e) {
logger.error("error while converting json to class", e);
}
}
return u;
}

Are line feeds allowed in JSON strings?

I need to send XML inside a JSON for my REST OSB 12c Proxy as follow:
{
"login": "jstein",
"identityContext": "jazn.com",
"taskId": "string",
"payload": {
"any_0": {
"any_01": "<afastamento xmlns:ns1='http: //www.tjsc.jus.br/soa/schemas/comagis/AfastamentoMagistrado' xsi:type='def: AfastamentoMagistradoType' xmlns:xsi='http: //www.w3.org/2001/XMLSchema-instance' xmlns='http: //xmlns.oracle.com/bpel/workflow/task'>
<ns1:Magistrado>719</ns1:Magistrado>
<ns1:Status>Inicial</ns1:Status>
<ns1:Vaga>8770</ns1:Vaga>
<ns1:Tipo>Licenca Nojo</ns1:Tipo>
<ns1:PeriodoReferencia/>
<ns1:DataInicialSolicitada>2015-10-10</ns1:DataInicialSolicitada>
<ns1:DataFinalSolicitada>2015-11-05</ns1:DataFinalSolicitada>
</afastamento>"
}
},
"outcome": "Start"
}
The OSB 12c send me back the error:
"errorMessage" : "ORABPEL-15235\n\nTranslation Failure.\nFailed to translate
JSON to XML. org.codehaus.jackson.JsonParseException: Illegal unquoted
character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be
included in string value\n at [Source: java.io.BufferedReader#7db921c7; line:
7, column: 619]\nThe incoming data does not conform to the NXSD schema. Please correct the problem.\n"
I am testing my JSON request at JSONLint, and it always gives me the error about start a String with <:
Parse error on line 7:
"any_01": "<afastamento xmlns:
-----------^
Expecting 'STRING, 'NUMBER, 'NULL', 'TRUE', FALSE', '{', '['
No, literal line feeds (CTRL-CHAR, code 10) and newlines are control characters that are not allowed within a JSON string:
XML does not require the line feeds between elements. You can simply remove them, changing your multi-line XML document to an equivalent single-line XML document that will be able to be passed as a JSON string without problem. Or, you may want to consider escaping the line feeds \n, or more generally, escaping the entire string:
How should I escape strings in JSON? [Java]
In C# how to encode XML to output it inside JSON in the JavaScript
part of a page

Importing JSON into R with in-line quotation marks

I'm attempting to read the following JSON file ("my_file.json") into R, which contains the following:
[{"id":"484","comment":"They call me "Bruce""}]
using the jsonlite package (0.9.12), the following fails:
library(jsonlite)
fromJSON(readLines('~/my_file.json'))
receiving an error:
"Error in parseJSON(txt) : lexical error: invalid char in json text.
84","comment":"They call me "Bruce""}]
(right here) ------^"
Here is the output from R escaping of the file:
readLines('~/my_file.json')
"[{\"id\":\"484\",\"comment\":\"They call me \"Bruce\"\"}]"
Removing the quotes around "Bruce" solves the problem, as in:
my_file.json
[{"id":"484","comment":"They call me Bruce"}]
But what is the issue with the escapement?
In R strings literals can be defined using single or double quotes.
e.g.
s1 <- 'hello'
s2 <- "world"
Of course, if you want to include double quotes inside a string literal defined using double quotes you need to escape (using backslash) the inner quotes, otherwise the R code parser won't be able to detect the end of the string correctly (the same holds for single quote).
e.g.
s1 <- "Hello, my name is \"John\""
If you print (using cat¹) this string on the console, or you write this string on a file you will get the actual "face" of the string, not the R literal representation, that is :
> cat("Hello, my name is \"John\"")
Hello, my name is "John"
The json parser, reads the actual "face" of the string, so, in your case json reads :
[{"id":"484","comment":"They call me "Bruce""}]
not (the R literal representation) :
"[{\"id\":\"484\",\"comment\":\"They call me \"Bruce\"\"}]"
That being said, also the json parser needs double-quotes escaping when you have quotes inside strings.
Hence, your string should be modified in this way :
[{"id":"484","comment":"They call me \"Bruce\""}]
If you simply modify your file by adding the backslashes you will be perfectly able to read the json.
Note that the corresponding R literal representation of that string would be :
"[{\"id\":\"484\",\"comment\":\"They call me \\\"Bruce\\\"\"}]"
in fact, this works :
> fromJSON("[{\"id\":\"484\",\"comment\":\"They call me \\\"Bruce\\\"\"}]")
id comment
1 484 They call me "Bruce"
¹
the default R print function (invoked also when you simply press ENTER on a value) returns the corresponding R string literal. If you want to print the actual string, you need to use print(quote=F,stringToPrint), or cat function.
EDIT (on #EngrStudent comment on the possibility to automatize quotes escaping) :
Json parser cannot do quotes escaping automatically.
I mean, try to put yourself in the computer's shoes and image you should parse this (unescaped) string as json: { "foo1" : " : "foo2" : "foo3" }
I see at least three possible escaping giving a valid json:
{ "foo1" : " : \"foo2\" : \"foo3" }
{ "foo1\" : " : "foo2\" : \"foo3" }
{ "foo1\" : \" : \"foo2" : "foo3" }
As you can see from this small example, escaping is really necessary to avoid ambiguities.
Maybe, if the string you want to escape has a really particular structure where you can recognize (without uncertainty) the double-quotes needing to be escaped, you can create your own automatic escaping procedure, but you need to start from scratch, because there's nothing built-in.