I have two json form. I tried to validate in jsonlint. It show error for first and it validate to second json.
Wrong json:
[ "name": {} ]
True json:
{ "name": {} }
Can any one explain why first one is wrong while second one true.
[ starts an array initializer. Valid entries are values separated by comments. Example:
["one", 2, "three"]
{ starts an object initializer. Valid entries are name/value pairs where each pair is a names in double quotes followed by a colon (:) followed by any valid value. Examples:
{"name": "value"}
{"name": {}}
{"name": ["one", 2, "three"]}
All of this is covered by the website and the standard.
Your first example is invalid because it's trying to define a name/value pair where a value is expected (in an array entry).
Related
I've got json file with some nested objects inside subs in it:
{
"version": 1,
"data": [{
"married": true,
"subs":[
{
"name":{
"sub1":{}
**},**
},
]
},
]}
If I add another 'name' object (with comma as separator), jsonDecode returns nothing.
if there goes single object, without comma - it's ok.
My Json structure is correct, and it's not restricted to use nested objects at all. Please anyone help.
This line has problem **},** and if changed to }, the problem will be solved..
you can check your json by online tools (e.g https://jsonformatter.curiousconcept.com/#)
Consider these two JSON files:
{
"tables": [
{
"columns": [
{
"name": "action",
"type": "string"
},
{
"name": "region",
"type": "string"
},
{
"name": "count_",
"type": "long"
},
{
"name": "min_timestamp",
"type": "datetime"
},
{
"name": "max_timestamp",
"type": "datetime"
}
],
"name": "PrimaryResult",
"rows": [
[
"ChangePassword",
"CN",
1,
"2022-07-19T11:51:52.8430729Z",
"2022-07-19T11:51:52.8430729Z"
]
]
}
]
}
{
"tables": [
{
"columns": [
{
"name": "action",
"type": "string"
},
{
"name": "region",
"type": "string"
},
{
"name": "count_",
"type": "long"
},
{
"name": "min_timestamp",
"type": "datetime"
},
{
"name": "max_timestamp",
"type": "datetime"
}
],
"name": "PrimaryResult",
"rows": [
[
"ChangePassword",
"CN",
1,
"2022-07-19T11:51:52.8430729Z",
"2022-07-19T11:51:52.8430729Z"
],
[
"Register",
"CN",
1,
"2022-07-19T11:51:52.8430729Z",
"2022-07-19T11:51:52.8430729Z"
]
]
}
]
}
They have the exact same schema and are almost the same. The difference is that first file has 1 row in the rows property, the second file has 2 rows in the rows property.
I use ConvertFrom-Json to load the files into PowerShell. When I start accessing .tables.rows[0], I expect that it will return the first row.
However, in the first file that has only 1 row, it actually returns the first column of the first row.
(gc file1.json | ConvertFrom-Json).tables.rows[0]
Outputs:
ChangePassword
If there are more than 1 row as in the second file, then the .tables.rows[0] behaves as expected.
(gc file2.json | ConvertFrom-Json).tables.rows[0]
Outputs:
ChangePassword
CN
1
2022-07-19T11:51:52.8430729Z
2022-07-19T11:51:52.8430729Z
How can I reliably process these JSON files in PowerShell regardless of if they have 1 row or multiple rows in them?
btw. These are actually JSON files produced by az cli as results from Azure Application Insights queries.
tl;dr
You need to avoid member-access enumeration if you want to access collection-valued properties as-is, which in your case means the following, as nimizen demonstrated in a comment:
# Note the [0] after .tables
# (-Raw was added for efficiency)
(gc -Raw file1.json | ConvertFrom-Json).tables[0].rows[0]
Because your tables property is itself an array, accessing just .tables results in member-access enumeration (even though the array happens to have just one element).
Using index [0] to target that one element explicitly allows access to its (one and only) .rows property without the latter's value being subject to implicit enumeration, as explained in the next section.
You're seeing a surprising aspect of PowerShell's member-access enumeration feature:
The values obtained from the members of a collection's elements are emitted as if they were sent to the pipeline, one by one, which has the following implications:
As happens by default in the pipeline, any such value is enumerated if it happens to be collection-valued, i.e. its elements are emitted, one by one.
On accessing the resulting output, if there's only one output object (whether or not that happens to be a collection itself), it is captured as-is; if there are two or more, they are implicitly collected in a regular PowerShell array (of type [object[]]).
The upshot for enumerated member values that happen to contain collections is:
The input collections themselves are always lost, because only their elements are output.
If there are multiple collection-valued member values, you'll end up with a single, flat array, invariably of type [object[]], that is the concatenation of the elements of all collections involved.
If there is only one collection-valued member value, and that value happens to be a single-element collection, you'll end up with that single element as-is (only with multiple elements would you end up with an [object[]] array).
This surprising behavior is the subject of GitHub issue #6802, though note that the behavior is unlikely to change, so as not to break backward compatibility.
A simplified example to demonstrate what happened in your case:
Note: JSON is incidental to the problem, so I'm using a [pscustomobject] instance below.
# Construct a single-element array that has a [pscustomobject] element.
# whose .prop property contains a single-element array whose only
# element is another array, typed [int[]].
$array = , [pscustomobject] #{ prop = , [int[]] (1, 2) }
# Use member-access enumeration to access the .prop member value
# of all elements in the array (here, there's only one), and get
# the first element.
$array.prop[0] # !! -> scalar 1, i.e. the first elem. of the *nested* array,
# !! because member-access enumeration *enumerated* the
# !! single-element outer array.
# Without member-access enumeration, the value of .prop is accessed as-is,
# as a nested array:
$array[0].prop[0] # -> [int[]] (1, 2), as expected.
don't know how to stop powershell behaving that way,
but got it "reliable" by
$test=Get-Item ".\file.json" | get-content | convertfrom-json
if($test.tables.rows[0].count -gt 1 ){
$result = $test.tables.rows[0][0]
}else{
$result = $test.tables.rows[0]
}
I have a json file with ElasticSearch events which can't be parsed by jq (funnily the json comes from jq) due to a missing comma. Here is an extract from the problematic place in the json file:
"end",
"protocol"
],
"dataset": "test",
"outcome": "success"
},
"#timestamp": "2020-08-23T04:47:10.000+02:00"
}
{
"agent": {
"hostname": "fb",
"type": "filebeat"
},
"destination": {
My jq command crashes at the closing brace (above "agent") as there is missing a comma after that brace (since a new event starts there). Now I know exactly the line and would like to add a comma there but couldn't find any options on how to do that efficiently. Since the file is around 25gb it is unsuitable to open it by nano or other tools. The error is parse error: Expected separator between values at line 192388762
Does anyone know if there is an efficient way to add a comma there so it looks like this?
"#timestamp": "2020-08-23T04:47:10.000+02:00"
},
{
"agent": {
Is there a command which I can tell to go to line X, column 1 and add a comma there (after column1)?
Are there brackets [] surrounding all these objects? If so, it is an array and there's missing commas indeed. But jq wouldn't have missed to produce them unless the previous filter was designed on purpose to behave that way. If there aren't surrounding brackets (which I presume according to the indentation of the sample), then it is a stream of objects that do not need a comma in between. In fact, putting a comma in between without the surrounding brackets would render the file inprocessible as it wouldn't be valid JSON anymore.
If it is a faulty array (the former case) maybe you're better off not using jq but rather a text stream editor such as sed or awk as you seem to know exactly where the commas are missing ("Is there a command which I can tell to go to line X, column 1 and add a comma there?")
If it is infact a stream of objects (the latter case), then you could use jq --slurp '…' or jq -n '[inputs] | …' to make it an array (surrounded by brackets and with commas in between) but the file (25 GB) has to fit entirely into your memory. If it doesn't, you need to use jq --stream '…' and handle the document (which has a different format then) according to the documentation for processing streams.
Illustrations:
This is an array of objects:
[
{"a": 1},
{"b": 2},
{"c": 3}
]
This is a stream of objects:
{"a": 1}
{"b": 2}
{"c": 3}
This is not valid JSON:
{"a": 1},
{"b": 2},
{"c": 3}
I want to split and transfer the json data in NiFi, Here is my json structure look like this;
I want to split json by id1,id2 array of json transfer to respective processor group say example processor_group a,b. I tried with evaluate json path $.id1,$.id2 but i didn't get exact solution. Can you please help me out from this issue;
{
"id1": [{
"u_name": "aa"
}, {
"addr": "bb"
}],
"id2": [{
"u_name": "aa"
}, {
"addr": "bb"
}]
}
The processor you're looking for is SplitJSON.
Configure it as follows:
Then, you'll receive two FlowFiles:
First one will contain the id1:
[{
"u_name": "aa"
}, {
"addr": "bb"
}]
second one will contain id2:
[{
"u_name": "aa"
}, {
"addr": "bb"
}]
Here is how to get to the values you want with EvaluateJsonPath:
#varun_rathinam Accessing json in an array object via EvaluateJsonPath can be quite confusing. I also notice the structure of your json is kind of confusing with same values in both. I have adjusted id2 for cc and dd for testing so that I can tell id1 and id2 values apart.
The solution you want is (see template for exact string values):
Notice we use the normal tree for each json object ( $.object ) then access the array ( 0, 1 ) then access the array's objects. Also notice it is possible to access the json object array with or without a . before the [.
Reference:
https://community.cloudera.com/t5/Support-Questions/how-to-extract-fields-in-flow-file-which-are-surrounded-by/m-p/208635
You can also find my template during testing of your issue on my GitHub:
https://github.com/steven-dfheinz/NiFi-Templates/blob/master/NiFI_EvaluateJsonPath_Demo.xml
I am trying to get json but am not sure how to read it. Here is the ouput of the json. Is quoteResponse an array or an object? also result is an array or an object? Thanks.
{
"quoteResponse": {
"result": [{
"language": "en-US",
"region": "US",
"quoteType": "EQUITY",
"quoteSourceName": "Delayed Quote",
"sourceInterval": 15,
"exchangeTimezoneName": "America/New_York",
"exchangeTimezoneShortName": "EDT",
"pageViews": {
"shortTermTrend": "UP",
"midTermTrend": "UP",
"longTermTrend": "UP"
},
"gmtOffSetMilliseconds": -14400000,
"marketState": "CLOSED",
"market": "us_market",
"priceHint": 2,
"esgPopulated": false,
"tradeable": true,
"triggerable": true,
"exchange": "NYQ",
"regularMarketPrice": 47.23,
"postMarketChangePercent": 0.0,
"postMarketTime": 1568407854,
"postMarketPrice": 47.23,
"postMarketChange": 0.0,
"regularMarketChangePercent": 0.66070205,
"exchangeDataDelayedBy": 0,
"regularMarketTime": 1568404920,
"regularMarketChange": 0.31000137,
"regularMarketVolume": 295978,
"regularMarketPreviousClose": 46.92,
"fullExchangeName": "NYSE",
"longName": "Cabot Corporation",
"shortName": "Cabot Corporation",
"symbol": "CBT"
}],
"error": null
}
}
Your quoteResponse is an object as
JSON objects are surrounded by curly braces {}.
They are written in key/value pairs.
Extract from your original JSON to have it directly in my answer for the purpose of comparison.
{
"quoteResponse": {
"result": [{
"language": "en-US"
"pageViews": {
"shortTermTrend": "UP"
}]
}
}
The result is just an array.
A bit offtopic, but maybe interesting:
I landed on a page which describe Javascript object literals and those are pretty similar to JSON but not the same. JSON derives from Javascript object literal syntax, so the syntax of both are very similar, one directly noticeable difference is that all names in JSON must be wrapped in double quotes.
JavaScript objects can also contain code (functions) and references to other objects. Their keys can be strings, numbers and symbols. etc.
This is an object which is shown by {}.
An array is contained in square brackets [].
You can also have an array of objects [{},{}] - just like the result in your example. You can keep on nesting.
quoteResponse is an object, its value is contained within braces {...}.
result is an array, it is contained within brackets [...].
quoteResponse is an object. It has properties result and error. In json object has following format {property: value, property2: value}
result is an array in json arrays has following format array: [{}, {}...]
quoteResponse is a JSONObject and result is a JSONArray. A JSONObject can be identified by the enclosing curly braces
{"quoteResponse":{"result":[{"language":"en-US","region":"US","symbol":"CBT"}],"error":null}}
Whereas a JSONArray can be identified by the enclosing box brackets as in result.
"result":[{"language":"en-US","region":"US","regularMarketChange":0.31000137,"regularMarketVolume":295978,"regularMarketPreviousClose":46.92,"fullExchangeName":"NYSE","longName":"Cabot Corporation","shortName":"Cabot Corporation","symbol":"CBT"}]