when reading logs, JSON changes from double quotes to single quotes - json

With the apache logs (below) I'm able to parse out a JSON:
[2014.02.14_21.24.22.543] other info I don't care about json: {
"petstore": "store_number_8",
"dogs":{
"terrier":{
"total":2
}
},
"cat":{
"siamese":{
"total":5
}
}
}
1) Is this valid JSON? 2) Why would the double quotes be turning into single quotes?
After I read it in, parse out the JSON, and display it I get the following:
{
'petstore': 'store_number_8',
'dogs':{
'terrier':{
'total':2
}
},
'cat':{
'siamese':{
'total':5
}
}
}
Btw, I'm using Node.js' fs.createStream to read in the logs, then simply doing a console out (so far I'm not doing any sanitization and eventually I will be writing it to a file).
fs.creatReadStream(logs).pipe(split()).on(data, function(line){
if(line.match(/json\:/)){
shouldThisBeValidJSON = JSON.parse(line.slice(line.indexOf('{'), line.length));
console.log(shouldThisBeValidJSON);
}
Thank you in advance.

console.log doesn't return JSON. It returns human-readable representation of this object.
So no, it's not JSON, it's something closer to JSON5.
If you want JSON to be displayed, you have to pass a string to console.log, i.e. console.log(JSON.stringify(shouldThisBeValidJSON))

Related

What is the ideal way to JSON.stringify in Terraform?

I'm working on my first Terraform project and I'm looking for the best way to stringify a JSON object. The resource I'm defining has a parameter that expects a JSON string. JSON structure is:
"document": {
"tag": "String Title",
"response": "There's a string response and perhaps a price like $[XX.XX]."
}
}
I don't think jsonencode or jsondecode do this. I could stringify them in advance but that isn't scalable in this case. I wasn't sure if I could do this with JavaScript or another language alongside Terraform, or if there's a function in HCL that will do it.
jsonencode in Terraform is exactly equivalent to JSON.stringify with only one argument in JavaScript.
For example, if you need to assign a string containing a JSON object to an argument called something_json, you could do that like this:
something_json = jsonencode({
document = {
tag = "String Title"
response = "There's a string response and perhaps a price like $[XX.XX]."
}
})
The above would set something_json to a minified version of the following JSON:
{
"document": {
"tag": "String Title",
"response": "There's a string response and perhaps a price like $[XX.XX]."
}
}
Terraform does not have an equivalent of the optional replacer and space arguments in JavaScript's JSON.stringify:
An equivalent of replacer isn't needed in Terraform because all possible values in the Terraform language have a defined JSON equivalent as described in the table in the jsonencode documentation.
space is for generating non-minified JSON; Terraform does not offer any way to do this because it is focused on generating JSON for machine consumption and so prefers to generate the most compact representation possible.

I want to write the XQuery to print the specific keys in JSON

This is my sample JSON
{
"id":"743",
"groupName":"group1",
"transation":{
"101":"success",
"102":"rejected",
"301":"processing"
}
}
Expected Result:
"101"
"102"
"301"
Can anyone please help me to print the above result using XQuery?
I can achieve this through JavaScript, but I need to write in XQuery.
Not knowing how you are reading the JSON document, whether as a doc in the database or parsing a JSON string, below uses xdmp:unquote() to parse a string, but you could instead just read the document from the database with fn:doc() or through cts:search().
Then, you could just XPath to the transation fields and return those node names with the name() function:
let $jsonData := xdmp:unquote('
{
"id":"743",
"groupName":"group1",
"transation":{
"101":"success",
"102":"rejected",
"301":"processing"
}
}')
return
$jsonData/transation/*/name()

How to pass JSON string in the JSON body of an HTTP request?

How can i pass a JSON string in the JSON body of the HTTP request? The request is sent by ServiceNow to Azure Devops to set the content of a pipeline variable.
The Json body is as below:
{
"resources":{
"repositories":{
"self":{
"refName":"refs/heads/master"
}
}
},
"variables":{
"request":{
"value":"{"key1": "value1"}" #here, i declare the json string
}
}
}
"{"key1": "value1"}" is the json string that i want to pass (this is just a sample of the string).
I have tried backslash '' in front of the braces. "\{"key1": "value1"\}" It didn't work.
I have tried to put the braces between single or double quotations. "'{'"key1": "value1"'}'" It didn't work.
Do you have any idea? Maybe it is doable with the ServiceNow's language but i am not expert of it. As Azure Devops only accepts strings as pipeline variables, i have to send the json as a string.
You have to escape the double quotes of the value:
{\"key1\": \"value1\"}

jackson jsonparser restart parsing in broken JSON

I am using Jackson to process JSON that comes in chunks in Hadoop. That means, they are big files that are cut up in blocks (in my problem it's 128M but it doesn't really matter).
For efficiency reasons, I need it to be streaming (not possible to build the whole tree in memory).
I am using a mixture of JsonParser and ObjectMapper to read from my input.
At the moment, I am using a custom InputFormat that is not splittable, so I can read my whole JSON.
The structure of the (valid) JSON is something like:
[ { "Rep":
{
"date":"2013-07-26 00:00:00",
"TBook":
[
{
"TBookC":"ABCD",
"Records":
[
{"TSSName":"AAA",
...
},
{"TSSName":"AAB",
...
},
{"TSSName":"ZZZ",
...
}
] } ] } } ]
The records I want to read in my RecordReader are the elements inside the "Records" element. The "..." means that there is more info there, which conforms my record.
If I have an only split, there is no problem at all.
I use a JsonParser for fine grain (headers and move to "Records" token) and then I use ObjectMapper and JsonParser to read records as Objects. For details:
configure(JsonParser.Feature.AUTO_CLOSE_SOURCE, false);
MappingJsonFactory factory = new MappingJsonFactory();
mapper = new ObjectMapper(factory);
mapper.configure(Feature.FAIL_ON_UNKNOWN_PROPERTIES,false);
mapper.configure(SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS,false);
parser = factory.createJsonParser(iStream);
mapper.readValue(parser, JsonNode.class);
Now, let's imagine I have a file with two inputsplits (i.e. there are a lot of elements in "Records").
The valid JSON starts on the first split, and I read and keep the headers (which I need for each record, in this case the "date" field).
The split would cut anywhere in the Records array. So let's assume I get a second split like this:
...
},
{"TSSName":"ZZZ",
...
},
{"TSSName":"ZZZ2",
...
}
] } ] } } ]
I can check before I start parsing, to move the InputStream (FSDataInputStream) to the beginning ("{" ) of the record with the next "TSSNAME" in it (and this is done OK). It's fine to discard the trailing "garbage" at the beginning. So we got this:
{"TSSName":"ZZZ",
...
},
{"TSSName":"ZZZ2",
...
},
...
] } ] } } ]
Then I handle it to the JsonParser/ObjectMapper pair seen above.
The first object "ZZZ" is read OK.
But for the next "ZZZ2", it breaks: the JSONParser complaints about malformed JSON. It is encountering a "," not being in an array. So it fails. And then I cannot keep on reading my records.
How could this problem be solved, so I can still be reading my records from the second (and nth) splits? How could I make the parser ignore these errors on the commas, or either let the parser know in advance it's reading contents of an array?
It seems it's OK just catching the exception: the parser goes on and it's able to keep on reading objects via the ObjectMapper.
I don't really like it - I would like an option where the parser could not throw Exceptions on nonstandard or even bad JSON. So I don't know if this fully answers the question, but I hope it helps.

Invalid JSON but validates on JSONLint

I have the following JSON which validates on JSONLint.com but when I pass it to JSON.parse() I get the error
SyntaxError: JSON.parse: unexpected character
...0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,...
This is apparently the last "correct": line
var theJSON = JSON.parse({
"data": [
{
"wrong": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
"correct": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
"wrong": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
"correct": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
"wrong": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
"correct": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
"wrong": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
"correct": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
}
]
});
You, like many, have confused JavaScript's literal syntax with JSON. This happens a lot as JSON uses a subset of JavaScript's literal syntax so it looks a lot alike. JSON, however, is always a string. It is a serialized data scheme for porting data structures between langs/platforms.
Also confusing is that a string of JSON which has been output by any platform can be copied and pasted right into JavaScript and used. Again, this is because of the shared syntax. Having pasted such output right into JavaScript, however, one is no longer using JSON--they are now writing JavaScript in literal syntax. That is, unless, you pasted it between quotes and properly escaped the resulting string. But there's no sense in doing so as then it needs to be parsed in order to end up with what you already had.
JSON.parse() is a method for unserializing data which had been serialized into JSON. It expects a string because, well, JSON is a string. You're passing an object (in literal syntax). It does not need parsing...it is already the thing you want.
Wrapping your object literal in single quotes would make the code work, but it would be pointless to do so as the parse would simply result in what you already have.
Your code would be better written if you replaced the variable named theJSON with one named theObject and made it look as such:
var theObject = {
data: [
{
wrong: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
correct: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
wrong: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
correct: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
wrong: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
correct: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
wrong: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
correct: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
}
]
};
Whatever code wanted to use the parse result should be fine once you've done it.