I have a API json response. The response has the same blocks type of data nested and i need to flatten this via the Azure datafactory. The depth of the children is variable. I'm not a expert in ADF and i couldn't find a example of how to fix this. I suspect that i need some recursive function to do this.
Some guidance would be very much appreciated.
Example json:
[
{
"id" : 1,
"name" : "item 1",
"children" : []
},
{
"id" : 2,
"name" : "item 2",
"children" : [
{
"id" : 3,
"name" : "item 3",
"children" : [
{
"id" : 4,
"name" : "item 4",
"children" : []
}
]
}
]
}
]
And i need to transform it into a sql table:
id
name
1
item 1
2
item 2
3
item 3
4
item 4
You will have to use Mapping data flow in Azure Data factory and use multiple Flatten transformations to get the desired output.
Related
How to define the json schema for array of json objects .
How to write schema for this json example
example :
[
{
"name" : "amy",
"Age" : "25"
},
{
"name" : "john",
"Age" : "20"
}
]
Tried in different ways but not working.
On https://jsonpath.com I have below example, using expression
$.phoneNumbers[?(#.id < 3)].number
on below JSON object.
{
"firstName": "John",
"lastName" : "doe",
"age" : 26,
"phoneNumbers": [
{
"type" : "iPhone",
"number": "0123-4567-1111",
"id": 1
},
{
"type" : "home",
"number": "0123-4567-2222",
"id": 2
},
{
"type" : "home",
"number": "0123-4567-3333",
"id": 3
}
]
}
Result is
[
"0123-4567-1111",
"0123-4567-2222"
]
Question
I only want the first string "0123-4567-1111", but appending [0] to my expression does not work. Expression $.phoneNumbers[?(#.id < 3)].number[0] gives result ["0","0"]. How can I get the first returned string?
Indeed you were very close to it by using this expression -
$.phoneNumbers[?(#.id < 3)].number[0]
In this expression you used id but in the json there is no id key so it resulted in undefined
try the expression like this using index -
$.phoneNumbers[0].number
It will return number from the first object of phoneNumbers list as : ["0123-4567-8888"]
If you want to go by conditional basis use the below expression which will return the number of type iphone -
$.phoneNumbers[?(#.type == 'iPhone')].number
Output -
["0123-4567-8888"]
I am using SwiftyJSON in Swift and I have three arrays that I would like to merge together into one array. If the name already exist, i'll like it to add the value to just get one element.
These are the arrays that I have:
var array1 = JSON(
"name" : "apple",
"Value" : 2
]
var array2 = JSON(
"name" : "apple",
"Value" : 4
]
var array3 = JSON(
"name" : "orange",
"Value" : 10
]
What is the best way to achieve the result below when I print the array?
{
{
"name": "apple"
"value": 6
} {
"name": "orange"
"value": 10
}
}
I'm trying to use JQ to find the most recent artifact in a Nexus API query. Right now, my JSON output looks something like:
{
"items" : [ {
"downloadUrl" : "https://nexus.ama.org/repository/Snapshots/org/sso/browser-manager/1.0-SNAPSHOT/browser-manager-1.0-20180703.144121-1.jar",
"path" : "org/sso/browser-manager/1.0-SNAPSHOT/browser-manager-1.0-20180703.144121-1.jar",
"id" : "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOTY1N2JiOTEyMTM1ZGRjZWQ",
"repository" : "Snapshots",
"format" : "maven2",
"checksum" : {
"sha1" : "7ac324905fb1ff15ef6020f256fcb5c9f54113ca",
"md5" : "bb25c483a183001dfdc58c07a71a98ed"
}
}, {
"downloadUrl" : "https://nexus.ama.org/repository/Snapshots/org/sso/browser-manager/1.0-SNAPSHOT/browser-manager-1.0-20180703.204941-2.jar",
"path" : "org/sso/browser-manager/1.0-SNAPSHOT/browser-manager-1.0-20180703.204941-2.jar",
"id" : "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOWM4YjQ0NmRjYzFkODkxM2U",
"repository" : "Snapshots",
"format" : "maven2",
"checksum" : {
"sha1" : "b4ba2049ea828391c720f49b6668a66a8b0bca9c",
"md5" : "6757c55c0e6d933dc90e398204cca966"
}
} ],
"continuationToken" : null
}
I've managed to use JQ to repackage the data as:
.items[] | { "id" : .id, "date" : (.path | scan("[0-9]{8}\\.[0-9-]*")) }
output:
{
"id": "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOTY1N2JiOTEyMTM1ZGRjZWQ",
"date": "20180703.144121-1"
}
{
"id": "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOWM4YjQ0NmRjYzFkODkxM2U",
"date": "20180703.204941-2"
}
Now I'm a little stuck trying to figure out which of the two JSON objects is the most recent. How can I sort by date and extract the id for that object?
Is there a better way to filter/sort this data? My example has only 2 items[] in the JSON response, but there may be a larger number of them.
The filter sort_by/1 will sort your timestamps in chronological order, but it requires an array as input, so you could write:
.items
| map({ "id" : .id, "date" : (.path | scan("[0-9]{8}\\.[0-9-]*")) })
| sort_by(.date)
| .[-1]
The trailing .[-1] selects the last item, so with your input the result would be:
{
"id": "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOWM4YjQ0NmRjYzFkODkxM2U",
"date": "20180703.204941-2"
}
I have a list of JSON objects, converted from the result of a SQL query. The JSON looks like this:
[ {
"CREATE_DATE_TIME" : "2018-02-04 11:00:03.0",
"EXTERNAL_ID" : "1111",
"CERT_NUMBER" : "123",
"DESCRIPTION" : "DESC 1",
"SOURCE_SYSTEM" : "WOULDIWAS"
}, {
"CREATE_DATE_TIME" : "2018-03-01 11:25:03.0",
"EXTERNAL_ID" : "2222",
"CERT_NUMBER" : "456",
"DESCRIPTION" : "DESC 2",
"SOURCE_SYSTEM" : "SHOOKSPEARE"
},
...
]
The output after JSON transform should be something like this:
{
"Jobs": [
{
"Notification": {
"ActivityDate" : "2018-02-04 11:00:03.0",
"ExternalId" : "1111",
"CertNum" : "123",
"Description" : "DESC 1",
"SourceSystem" : "WOULDIWAS",
"RecordType" : "Task Notification"
}, {
"Notification": {
"ActivityDate" : "2018-03-01 11:25:03.0",
"ExternalId" : "2222",
"CertNum" : "456",
"Description" : "DESC 2",
"SourceSystem" : "SHOOKSPEARE",
"RecordType" : "Task Notification"
},
...
]
}
(The RecordType is a literal string, not derived from the input JSON)
Each row / entry (JSON object enclosed in {}) in the input JSON is guaranteed to be unique but there is no key here that would indicate that. The row / entry in the input should correspond 1-1 to { Notification: {...} } in the output. How should I construct my Jolt Spec to do this?
Not to sound offending or anything, but you should've posted what you've already tried.
Anyway here's the spec to get your intended output format
[
{
"operation": "shift",
"spec": {
"*": "Jobs[].Notification"
}
}
]
I would suggest you try out renaming the fields yourself, because practicing JOLT is the best way to learn
If you still need help, I'll complete the answer for you.
Here's a few reading material Documentation, the Slide deck.
And you can learn a lot from the issues page where Milo Simpson has already solved queries for most of your questions.