Delete duplications in JSON file - json

I am trying to reedit json file to print only subgroups that has any attributes marked as "change": false.
Json below:
{"group":{
"subgroup1":{
"attributes":[
{
"change":false,
"name":"Name"},
{
"change":false,
"name":"SecondName"},
],
"id":1,
"name":"MasterTest"},
"subgroup2":{
"attributes":[
{
"change":true,
"name":"Name"
},
{
"change":false,
"name":"Newname"
}
],
"id":2,
"name":"MasterSet"},
}}
I was trying to use command:
cat test.json | jq '.group[] | select (.attributes[].change==false)
which produce needed output but with duplicates. Can anyone help here? Or shall I use different command to achieve that result?

.attributes[] iterates over the attributes, and each iteration step produces its own result. Use the any filter which aggregates multiple values into one, in this case a boolean with the meaning of "at least one":
.group[] | select(any(.attributes[]; .change==false))
{
"attributes": [
{
"change": false,
"name": "Name"
},
{
"change": false,
"name": "SecondName"
}
],
"id": 1,
"name": "MasterTest"
}
{
"attributes": [
{
"change": true,
"name": "Name"
},
{
"change": false,
"name": "Newname"
}
],
"id": 2,
"name": "MasterSet"
}
Demo

Looks to me like the duplicate is NOT a duplicate, but a condition arising from a nested sub-grouping, which gives the appearance of a duplicate. You should look to see if there is a switch to skip processing sub-groups when the upper-level meets the condition, thereby avoiding the perceived duplication.

Related

How to operate with json list in CMake?

I have the following code which I'm trying to read in CMake.
{
"demo": [
{
"name": "foo0",
"url": "url1",
"verify_ssl": true
},
{
"name": "foo1",
"url": "url1",
"verify_ssl": true
},
{
"name": "foo2",
"url": "url2",
"verify_ssl": true
}
]
}
I'm trying to access a member from the list above, for example demo[0].name without success, what I'm doing wrong?
file(READ "${CONAN_CACHE}/demo.json" MY_JSON_STRING)
string(JSON CUR_NAME GET ${MY_JSON_STRING} demo[0].name)
One at a time.
string(JSON CUR_NAME GET ${MY_JSON_STRING} demo 0 name)

jOOQ JSON formatting as array of objects

I have the following (simplified) jOOQ query:
val result = context.select(
jsonObject(
key("id").value(ITEM.ID),
key("title").value(ITEM.NAAM),
key("resources").value(
jsonArrayAgg(ITEM_INHOUD.RESOURCE_ID).absentOnNull()
)
)
).from(ITEM).fetch()
Now the output that I want is:
[
{
"id": "0da04cc5-f70c-4fb3-b5c7-dc645d342631",
"title": "Title1",
"resources": [
"8b0f6d5c-67fc-47ca-be77-d1735e7721ce",
"ea0316db-1cfd-46d7-8260-5c1a4e65a0cd"
]
},
{
"id": "0f7e67e6-5187-47e2-9f1d-dab08feba38b",
"title": "Title2"
}
]
result.formtJSON() gives the following output:
{
"fields": [
{
"name": "json_object",
"type": "JSON"
}
],
"records": [
[
{
"id": "0da04cc5-f70c-4fb3-b5c7-dc645d342631",
"title": "Title 1"
}
]
]
}
Disabling the headers with result.formatJSON(JSONFormat.DEFAULT_FOR_RECORDS) will get me:
[
[
{
"id": "0da04cc5-f70c-4fb3-b5c7-dc645d342631",
"title": "Title1",
"resources": [
"8b0f6d5c-67fc-47ca-be77-d1735e7721ce",
"ea0316db-1cfd-46d7-8260-5c1a4e65a0cd"
]
}
],
[
{
"id": "0f7e67e6-5187-47e2-9f1d-dab08feba38b",
"title": "Title2"
}
]
]
where I don't want the extra array.
Further customizing the JSONformatter with result.formatJSON(JSONFormat().header(false).recordFormat(JSONFormat.RecordFormat.OBJECT)) I get:
[
{
"json_object": {
"id": "0da04cc5-f70c-4fb3-b5c7-dc645d342631",
"title": "Title1",
"resources": [
"8b0f6d5c-67fc-47ca-be77-d1735e7721ce",
"ea0316db-1cfd-46d7-8260-5c1a4e65a0cd"
]
}
},
{
"json_object": {
"id": "0f7e67e6-5187-47e2-9f1d-dab08feba38b",
"title": "Title2"
}
}
]
where I don't want the object wrapped in json_object.
Is there a way to get the output I want?
Doing it with Result.formatJSON()
This is clearly a flaw in the jOOQ 3.14.0 implementation of Result.formatJSON(). In the special case where there is only one column, and that column is of type JSON or JSONB, the column name may not really matter, and thus its contents should be flattened into the object describing the row. I've created a feature request for this: https://github.com/jOOQ/jOOQ/issues/10953. It will be available in jOOQ 3.15.0 and 3.14.4. You will be able to do this:
result.formatJSON(JSONFormat().header(false).wrapSingleColumnRecords(false));
The RecordFormat is irrelevant here. This works the same way for RecordFormat.ARRAY and RecordFormat.OBJECT
Doing it directly with SQL
Of course, you can always work around this by moving all the logic into SQL. You probably simplified your query by omitting a JOIN and GROUP BY. I'm assuming this is equivalent to what you want:
JSON result = context.select(
jsonArrayAgg(jsonObject(
key("id").value(ITEM.ID),
key("title").value(ITEM.NAAM),
key("resources").value(
select(jsonArrayAgg(ITEM_INHOUD.RESOURCE_ID).absentOnNull())
.from(ITEM_INHOUD)
.where(ITEM_INHOUD.ITEM_ID.eq(ITEM.ID))
)
))
).from(ITEM).fetchSingle().value1()
Note that JSON_ARRAYAGG() aggregates empty sets into NULL, not into an empty []. If that's a problem, use COALESCE()

In Logic Apps JSON Array while parsing throwing error for single object but for multiple objects it is working fine

While parsing JSON in Azure Logic App in my array I can get single or multiple values/objects (Box as shown in below example)
Both type of inputs are correct but when only single object is coming then it is throwing an error "Invalid type. Expected Array but got Object "
Input 1 (Throwing error) : -
{
"MyBoxCollection":
{
"Box":{
"BoxName": "Box 1"
}
}
}
Input 2 (Working Fine) : -
{
"MyBoxCollection":
[
{
"Box":{
"BoxName": "Box 1"
},
"Box":{
"BoxName": "Box 2"
}
}]
}
JSON Schema :
"MyBoxCollection": {
"type": "object",
"properties": {
"box": {
"type": "array",
items": {
"type": "object",
"properties": {
"BoxName": {
"type": "string"
},
......
.....
..
}
Error Details :-
[
{
"message": "Invalid type. Expected Array but got Object .",
"lineNumber": 0,
"linePosition": 0,
"path": "Order.MyBoxCollection.Box",
"schemaId": "#/properties/Root/properties/MyBoxCollection/properties/Box",
"errorType": "type",
"childErrors": []
}
]
I used to use the trick of injecting a couple of dummy rows in the resultset as suggested by the other posts, but I recently found a better way. Kudos to Thomas Prokov for providing the inspiration in his NETWORG blog post.
The JSON parse schema accepts multiple choices as type, so simply replace
"type": "array"
with
"type": ["array","object"]
and your parse step will happily parse either an array or a single value (or no value at all).
You may then need to identify which scenario you're in: 0, 1 or multiple records in the resultset? I'm pasting below how you can create a variable (ResultsetSize) which takes one of 3 values (rs_0, rs_1 or rs_n) for your switch:
"Initialize_ResultsetSize": {
"inputs": {
"variables": [
{
"name": "ResultsetSize",
"type": "string",
"value": "rs_n"
}
]
},
"runAfter": {
"<replace_with_name_of_previous_action>": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Check_if_resultset_is_0_or_1_records": {
"actions": {
"Set_ResultsetSize_to_0": {
"inputs": {
"name": "ResultsetSize",
"value": "rs_0"
},
"runAfter": {},
"type": "SetVariable"
}
},
"else": {
"actions": {
"Set_ResultsetSize_to_1": {
"inputs": {
"name": "ResultsetSize",
"value": "rs_1"
},
"runAfter": {},
"type": "SetVariable"
}
}
},
"expression": {
"and": [
{
"equals": [
"#string(body('<replace_with_name_of_Parse_JSON_action>')?['<replace_with_name_of_root_element>']?['<replace_with_name_of_list_container_element>']?['<replace_with_name_of_item_element>']?['<replace_with_non_null_element_or_attribute>'])",
""
]
}
]
},
"runAfter": {
"Initialize_ResultsetSize": [
"Succeeded"
]
},
"type": "If"
},
"Process_resultset_depending_on_ResultsetSize": {
"cases": {
"Case_no_record": {
"actions": {
},
"case": "rs_0"
},
"Case_one_record_only": {
"actions": {
},
"case": "rs_1"
}
},
"default": {
"actions": {
}
},
"expression": "#variables('ResultsetSize')",
"runAfter": {
"Check_if_resultset_is_0_or_1_records": [
"Succeeded",
"Failed",
"Skipped",
"TimedOut"
]
},
"type": "Switch"
}
For this problem, I met another stack overflow post which is similar to this problem. While there is one "Box", it will be shown as {key/value pair} but not [array] when we convert it to json format. I think it is caused by design, so maybe we can just add a record "Box" at the source of your xml data such as:
<Box>specific_test</Box>
And do some operation to escape the "specific_test" in the next steps.
Another workaround for your reference:
If your json data has only one array, we can use it to do a judgment. We can judge the json data if it contains "[" character. If it contains "[", the return value is the index of the "[" character. If not contains, the return value is -1.
The expression shows as below:
indexOf('{"MyBoxCollection":{"Box":[aaa,bbb]}}', '[')
The screenshot above is the situation when it doesn't contain "[", it return -1.
Then we can add a "If" condition. If >0, do "Parse JSON" with one of the schema. If =-1, do "Parse JSON" with the other schema.
Hope it would be helpful to your problem~
We faced a similar issue. The only solution we find is by manipulating the XML before conversion. We updated XML nodes which needs to be an array even when we have single element using this. We used a Azure function to update the required XML attributes and then returned the XML for conversion in Logic Apps. Hope this helps someone.

Hive Sql Query To get Json Object from Json Array

I have a json inside 'content' column in the following format:
{ "identifier": [
{
"type": {
"coding": [
{
"code": "MRN",
}
]
},
"value": "181"
},
{
"type": {
"coding": [
{
"code": "PID",
}
]
},
"value": "5d3669b0"
},
{
"type": {
"coding": [
{
"code": "IPN",
}
]
},
"value": "41806"
}
]}
I have to run an hive query to get the "value" of the code which is equal to "MRN".
I have written the following query but its not giving the value as expected:
select get_json_object(content,'$.identifier.value')as Mrn from Doctor where get_json_object(content,'$.identifier.type.coding.code') like '%MRN%'
I dont want to give particular array position like:
select get_json_object(content,'$.identifier[0].value')as Mrn from Doctor where get_json_object(content,'$.identifier[0].type.coding.code') like '%MRN%'
As the json gets created randomly and the position is not fixed always.
Give [ * ] to avoid giving position.
select get_json_object(content,'$.identifier[*].value')as Mrn from Doctor where get_json_object(content,'$.identifier[*].type.coding.code') like '%MRN%'

How to get specified values from ansbile facts

I'm confused a little bit.
I alredy tried jmespath, but didn't help.
I need to get a high-level object which contains a specified object.
In the example below we see an ansible fact about HDDs. I need to get the disk name (sdf, sdg or dm-0) of the HDD which contains a specified partition, e.g. sdf1.
I've got this jmespath query:
msg.*.[partitions.sdf1]
but it just shows me all inside sdf1. Filters like [?partitions=="sdf1"] don't work here,
so the question is: how to preserve the whole sdf object in my example?
thanks in advance!
{
"msg": {
"sdf": {
"partitions": {
"sdf1": {
"holders": [],
"links": {
"ids": [
"17101686F123-part1",
"wwn-0x123456-part1"
]
},
"sectors": "1875380224"
}
},
"removable": "0",
"rotational": "0"
},
"sdg": {
"partitions": {
"sdg1": {
"holders": [],
"links": {
"ids": [
"164414123CEB-part1",
"wwn-0x1233451234831ceb-part1"
]
},
"uuid": "F301-FA7F"
}
},
"removable": "0"
},
"dm-0": {
"holders": [],
"host": "",
"links": {
"ids": [],
"uuids": []
},
"vendor": null,
"virtual": 1
}
}
}
This should work:
msg.*.{value: #, condition: partitions.sdf1}[?condition].value
explanation
At first we create a temporary object For each HDD object: {value, condition}. condition will be null if partitions.sdf1 doesn't exist on the corresponding HDD (sdf, sdg dm-0) object. If partitions.sdf1 exists, condition will contain that object, that is:
"condition": {
"holders": [],
"links": {
"ids": [
"17101686F123-part1",
"wwn-0x123456-part1"
]
},
"sectors": "1875380224"
}
Using [?condition] you filter out all objects where condition is null. Finally, we extract the actual value using .value