I'm trying to transform a record json into individual json records but have a hard time getting the values rather than the names. I expect the keys to change on occasion and there to be more than 4 records on occasion so I wanted it to be dynamic. It seems the current transform will only give me the keys and not the values broken out into there own records.
Input
[
{
"Owner": {
"0": "CIMections",
"1": "CIMections",
"2": "CIMections",
"3": "CIMections"
},
"Name": {
"0": "AFE 20NSF044",
"1": "AFE 20NSF044",
"2": "AFE 20NSF044",
"3": "AFE 20NSF044"
},
"Producer": {
"0": "Produtream",
"1": "Produtream",
"2": "Produtream",
"3": "Produtream"
},
"Producers ID": {
"0": "NTI XR 001-004",
"1": "NTI XR 001-004",
"2": "NTI XR 001-004",
"3": "NTI XR 001-004"
},
"Weld - Real Time Count": {
"0": "",
"1": "",
"2": "",
"3": ""
},
"Character Set": {
"0": "ISO_IR 192",
"1": "ISO_IR 192",
"2": "ISO_IR 192",
"3": "ISO_IR 192"
},
"inv# Welds": {
"0": "Accepted 001",
"1": "Accepted 002",
"2": "Accepted 003",
"3": "Accepted 004"
},
"invoice": {
"0": 893300361,
"1": 411904740,
"2": 673190473,
"3": 1426231494
},
"status": {
"0": "Done",
"1": "Done",
"2": "Done",
"3": "Done"
},
"Date Completed": {
"0": "20210301 163500.000000",
"1": "20210301 163500.000000",
"2": "20210301 163500.000000",
"3": "20210301 163500.000000"
},
"Institution Name": {
"0": "NXXT Digital",
"1": "NXXT Digital",
"2": "NXXT Digital",
"3": "NXXT Digital"
},
"file_id": {
"0": "00001",
"1": "00002",
"2": "00003",
"3": "00004"
}
}
]
Current Transformation
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"Owner": "Owner.[].#(1,0)",
"$": "&",
"#": "#clientId"
}
}
}
}
]
Im Getting
{
"Owner" : "Owner",
"Name" : "Name",
"Producer" : "Producer",
"Producers ID" : "Producers ID",
"Weld - Real Time Count" : "Weld - Real Time Count",
"Character Set" : "Character Set",
"inv# Welds" : "inv# Welds",
"invoice" : "invoice",
"status" : "status",
"Date Completed" : "Date Completed",
"Institution Name" : "Institution Name",
"file_id" : "file_id"
}
But wanting
{
"Owner" : "CIMections",
"Name" : "AFE 20NSF044",
"Producer" : "Produtream",
"Producers ID" : "NTI XR 001-004",
"Weld - Real Time Count" : "",
"Character Set" : "ISO_IR 192",
"inv# Welds" : "Accepted 001",
"invoice" : "893300361",
"status" : "Done",
"Date Completed" : "20210301 163500.000000",
"Institution Name" : "NXXT Digital",
"file_id" : "00001"
},
{
"Owner" : "CIMections",
"Name" : "AFE 20NSF044",
"Producer" : "Produtream",
"Producers ID" : "NTI XR 001-004",
"Weld - Real Time Count" : "",
"Character Set" : "ISO_I 192",
"inv# Welds" : "Accepted 002",
"invoice" : "411904740",
"status" : "Done",
"Date Completed" : "20210301 163500.000000",
"Institution Name" : "NXXT Digital",
"file_id" : "00002"
}...
$ wildcard is not needed, but only using # wildcard is enough, along with filtering by indices with 0 and 1 (0|1) such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"0|1": {
"#": "[&].&2"
}
}
}
}
}
]
where prepending the value with [&]. converts each individual array to seperate objects within a single array.
If you need to get all the objects within the array, then replace "0|1" with "*" wildcard.
Related
Given JSON:
[
{
"1": "false"
"2": "true",
"3": "true"
},
{
"1": "false"
"2": "false",
"3": "false"
},
{
"1": "true"
"2": "true",
"3": "true"
}
]
Given array as an argument passed to jq:
["1","2","3"]
Need to remove JSON element using jq tool if at least one key is not "true".
Desired output:
[
{
"1": "false"
"2": "true",
"3": "true"
},
{
"1": "true"
"2": "true",
"3": "true"
}
]
Given JSON:
[
{
"1": "false"
"2": "true",
"3": "true"
},
{
"1": "false"
"2": "false",
"3": "false"
},
{
"1": "true"
"2": "true",
"3": "true"
}
]
Given array as an argument passed to jq:
["1","2","3"]
Need to remove JSON element using jq tool if at least one key is not "true".
Desired output:
[
{
"1": "false"
"2": "true",
"3": "true"
},
{
"1": "true"
"2": "true",
"3": "true"
}
]
I don't know what role the argument ["1","2","3"] should play, but apart from that, you can convert the object items from string to boolean using fromjson, then aggregate them appropriately, and use this result in a map(select(…)) clause to filter the input array.
Now, if you want to remove items
if at least one key is not "true"
meaning you want to keep only those that are entirely true, use the all aggregator:
jq 'map(select(map(fromjson) | all))'
[
{
"1": "true",
"2": "true",
"3": "true"
}
]
Demo
But if, as your desired output suggests, you want to remove those items that are entirely false, meaning to keep only those where there is at least one value which is true, use the any aggregator:
jq 'map(select(map(fromjson) | any))'
[
{
"1": "false",
"2": "true",
"3": "true"
},
{
"1": "true",
"2": "true",
"3": "true"
}
]
Demo
I am currently stuck trying to convert JSON strings into a useable format.
The data was originally in a pandas dataframe, and I have it stored in JSON format in a azure blob storage container. I then used this Compose base64(outputs('Get_blob_content_(V2)')?['body']) , then used
base64ToString(outputs('Compose')) to push the JSON file into string data type.
Afterwards I have parsed it using Parse JSON function.
currently when I email it, I'm getting a list of sorts of the JSON entries. I would like to put this into a tabular format or table format. I have tried both the CSV and HTML function options but neither do anything.
After reproducing from our end, we could able to achieve your requirement using the below flow.
Below is the json I'm working on
{
"0": "2022-08-12 16:00:00",
"1": "2022-09-05 16:00:00",
"2": "2022-09-05 16:00:00",
"3": "2022-09-01 16:00:00",
"4": "2022-09-03 16:00:00",
"5": "2022-09-21 16:00:00",
"6": "2022-09-23 16:00:00",
"7": "2022-09-29 10:00:00",
"8": "2022-10-07 16:00:00",
"9": "2022-09-30 16:00:00",
"10": "2022-10-07 17:00:00",
"11": "2022-10-21 16:00:00"
}
In Create HTML table I have actually used the Parse JSON's results.
RESULTS:
Below is the successful run in my logic app
In my mail
Here is the code view of my logic app
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Compose": {
"inputs": {
"0": "2022-08-12 16:00:00",
"1": "2022-09-05 16:00:00",
"2": "2022-09-05 16:00:00",
"3": "2022-09-01 16:00:00",
"4": "2022-09-03 16:00:00",
"5": "2022-09-21 16:00:00",
"6": "2022-09-23 16:00:00",
"7": "2022-09-29 10:00:00",
"8": "2022-10-07 16:00:00",
"9": "2022-09-30 16:00:00",
"10": "2022-10-07 17:00:00",
"11": "2022-10-21 16:00:00"
},
"runAfter": {},
"type": "Compose"
},
"Create_HTML_table": {
"inputs": {
"columns": [
{
"header": "0",
"value": "#body('Parse_JSON')?['0']"
},
{
"header": "1",
"value": "#body('Parse_JSON')?['1']"
},
{
"header": "2",
"value": "#body('Parse_JSON')?['2']"
},
{
"header": "3",
"value": "#body('Parse_JSON')?['3']"
},
{
"header": "4",
"value": "#body('Parse_JSON')?['4']"
},
{
"header": "5",
"value": "#body('Parse_JSON')?['5']"
},
{
"header": "6",
"value": "#body('Parse_JSON')?['6']"
},
{
"header": "7",
"value": "#body('Parse_JSON')?['7']"
},
{
"header": "8",
"value": "#body('Parse_JSON')?['8']"
},
{
"header": "9",
"value": "#body('Parse_JSON')?['9']"
},
{
"header": "10",
"value": "#body('Parse_JSON')?['10']"
},
{
"header": "11",
"value": "#body('Parse_JSON')?['11']"
}
],
"format": "HTML",
"from": "#array(outputs('Compose'))"
},
"runAfter": {
"Parse_JSON": [
"Succeeded"
]
},
"type": "Table"
},
"Parse_JSON": {
"inputs": {
"content": "#outputs('Compose')",
"schema": {
"properties": {
"0": {
"type": "string"
},
"1": {
"type": "string"
},
"2": {
"type": "string"
},
"3": {
"type": "string"
},
"4": {
"type": "string"
},
"5": {
"type": "string"
},
"6": {
"type": "string"
},
"7": {
"type": "string"
},
"8": {
"type": "string"
},
"9": {
"type": "string"
},
"10": {
"type": "string"
},
"11": {
"type": "string"
}
},
"type": "object"
}
},
"runAfter": {
"Compose": [
"Succeeded"
]
},
"type": "ParseJson"
},
"Send_an_email_(V2)": {
"inputs": {
"body": {
"Body": "<p>#{body('Create_HTML_table')}</p>",
"Importance": "Normal",
"Subject": "HTML Table",
"To": "v-swethaka#microsoft.com"
},
"host": {
"connection": {
"name": "#parameters('$connections')['office365']['connectionId']"
}
},
"method": "post",
"path": "/v2/Mail"
},
"runAfter": {
"Create_HTML_table": [
"Succeeded"
]
},
"type": "ApiConnection"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"manual": {
"inputs": {
"schema": {}
},
"kind": "Http",
"type": "Request"
}
}
},
"parameters": {
"$connections": {
"value": {
"office365": {
"connectionId": "/subscriptions/<SubId>/resourceGroups/<RG>/providers/Microsoft.Web/connections/office365",
"connectionName": "office365",
"id": "/subscriptions/<SubId>/providers/Microsoft.Web/locations/centralus/managedApis/office365"
}
}
}
}
}
I have a data set which I am trying to group by two separate keys, but nested. I have tried to use the groupBy method but I am then having a hard time with the sum within the keys.
I have also tries to use the pluck method but that also seems to become more challenging with nested groupBy. Any suggestions would be welcomed.
Input:
var sales = [{
"price": "500" as Number,
"store": "123",
"category": "1"
},
{
"price": "400" as Number,
"store": "123",
"category": "2"
},
{
"price": "700" as Number,
"store": "123",
"category": "2"
},
{
"price": "800" as Number,
"store": "456",
"category": "6"
},
{
"price": "900" as Number,
"store": "456",
"category": "7"
},
{
"price": "400" as Number,
"store": "456",
"category": "6"
}
]
Output:
[
{
"123": {
"1": {"price": "500"},
"2": {"price": "1100"}
}
},
{
"456": {
"6": {"price": "800"},
"7": {"price": "1300"}
}
}
]
There may be an easiest way. I don't understand why the prices are stored as strings instead of numbers in both input and output. It would simplify the output at least to use numbers.
%dw 2.0
output application/json
var sales = [{
"price": "500" as Number,
"store": "123",
"category": "1"
},
{
"price": "400" as Number,
"store": "123",
"category": "2"
},
{
"price": "700" as Number,
"store": "123",
"category": "2"
},
{
"price": "800" as Number,
"store": "456",
"category": "6"
},
{
"price": "900" as Number,
"store": "456",
"category": "7"
},
{
"price": "400" as Number,
"store": "456",
"category": "6"
}
]
fun sumCategories(a)=
a groupBy ($.category)
mapObject ((value, key, index) -> {
(key):
value reduce ((item, accumulator={ price: 0}) -> {price: item.price + accumulator.price} )
mapObject {price: $ as String} // assumes single attribute
})
---
sales
groupBy ($.store)
mapObject ((value, key, index) -> (key): sumCategories(value))
pluck {($$):$}
Output:
[
{
"123": {
"1": {
"price": "500"
},
"2": {
"price": "1100"
}
}
},
{
"456": {
"6": {
"price": "1200"
},
"7": {
"price": "900"
}
}
}
]
I'm trying to transform JSON array into elements are objects with the specified key/value pairs
{
"Resource": "sorPersonRole",
"Roleid": "1",
"Timestamp": "2010-06-30 00:00:00.0",
"Release": "Public",
"DOB": "2064-09-05",
"Active": "Y",
"enterprise_id": "a12s33",
"Inactive_enterprise_id": "",
"emp_ID": "123456",
"Inactive_emp_id": "000821972",
"Username": "",
"A_ID": "fsgf1234jhgfs3",
"P_ID": "w123456",
"Is Email Valid": "Y",
"Flag": "N",
"Registered": "spring",
"Description": "mainland corp",
"End Date": null
}
Expected output:
{
"meta" : {
"Resource" : "sorPersonRole",
"Roleid" : "1",
"Timestamp" : "2010-06-30 00:00:00.0",
"Release" : "Public",
"Sorid" : "w123456"
},
"sorAttributes" : {
"DOB" : "2064-09-05",
"Active" : "Y",
"End Date" : null,
"identifiers":
[
{
"type" : "enterprise"
"enterprise_id" : "a12s33",
"Username" : ""
},
{
"type" : "former-enterprise"
"Inactive_enterprise_id" : ""
},
{
"type" : "UID"
"emp_ID" : "123456",
"Inactive_emp_id" : "000821972"
},
{
"type" : "National"
"A_ID" : "fsgf1234jhgfs3"
}
],
"mainLand:com:adhoc" : {
"Is Email Valid" : "Y",
"Flag" : "N",
"Registered" : "spring",
"Description" : "mainland corp"
}
}
}
current Jolt spec: which I am not getting desired output
[
{
"operation": "shift",
"spec": {
"Resource": "meta.&",
"P_ID": "meta.Sorid",
"Roleid": "meta.&",
"Timestamp": "meta.&",
"Release": "meta.&",
"enterprise_id": "sorAttributes.Identifiers.type.enterprise.&",
"Inactive_enterprise_id": "sorAttributes.Identifiers.type.former-enterprise.&",
"emp_ID": "sorAttributes.Identifiers.type.UID.&",
"Inactive_emp_id": "sorAttributes.Identifiers.type.UID.&",
"Username": "sorAttributes.Identifiers.type.enterprise.&",
"A_ID": "sorAttributes.Identifiers.type.National.&",
"Is Email Valid": "sorAttributes.mainLand:com:adhoc.&",
"Flag": "sorAttributes.mainLand:com:adhoc.&",
"Registered": "sorAttributes.mainLand:com:adhoc.&",
"Description": "sorAttributes.mainLand:com:adhoc.&",
"*": "sorAttributes.&"
}
}
]
I have tried the different JsonSpecs provided on different websites, could able to match expected output. Also tried using two-shift operations but no luck, Any help or suggestion will be appreciated.
Thanks.
This can help,
For the nodes to be shifted into the identifier array, shift one level more.
[
{
"operation": "shift",
"spec": {
"Resource": "meta.&",
"Roleid": "meta.&",
"Timestamp": "meta.&",
"Release": "meta.&",
"P_ID": "meta.Sorid",
"DOB": "sorAttributes.&",
"Active": "sorAttributes.&",
"End Date": "sorAttributes.&",
"Is Email Valid": "sorAttributes.mainLand:com:adhoc.&",
"Flag": "sorAttributes.mainLand:com:adhoc.&",
"Registered": "sorAttributes.mainLand:com:adhoc.&",
"Description": "sorAttributes.mainLand:com:adhoc.&",
"enterprise_id": {
"#enterprise": "sorAttributes.identifiers[#2].type",
"#": "sorAttributes.identifiers[#2].&",
"#(1,Username)": "sorAttributes.identifiers[#2].Username"
},
"Inactive_enterprise_id": {
"#former-enterprise": "sorAttributes.identifiers[#2].type",
"#": "sorAttributes.identifiers[#2].&"
},
"Inactive_emp_id": {
"#UID": "sorAttributes.identifiers[#2].type",
"#": "sorAttributes.identifiers[#2].&",
"#(1,emp_ID)": "sorAttributes.identifiers[#2].emp_ID"
},
"A_ID": {
"#National": "sorAttributes.identifiers[#2].type",
"#": "sorAttributes.identifiers[#2].&"
}
}
}, {
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
}
]
I want to create a nested JSON for using it as a Post method parameter. I tried NSDictionary, Array, [String:Any], etc. But failed to get the accurate JSON structure.
I want a JSON format, like this
{
"Command": [
{
"commandData": {
"name": "name",
"description": "description",
"title": "title",
"actions": [
{
"ERR": [
{
"ERR_LINE": {
"Value": {
"1": 1.0,
"2": 1.0,
"3": 1.0,
"4": 1.0,
"5": 1.0,
"6": 1.0
}
}
}
]
}
]
},
"errID": "id",
"issueID": "id"
}
]
}
You can do like this:
var postData = Dictionary<String,Any>()
postData = [
"Command": [
"commandData": [
"name": "name",
"description": "description",
"title": "title",
"actions": [
"ERR": [
"ERR_LINE": [
"Value": [
"1": 1.0,
"2": 1.0,
"3": 1.0,
"4": 1.0,
"5": 1.0,
"6": 1.0
]
]
]
]
],
"errID": "id",
"issueID": "id"
]
]
Hope it works.