I am attempting to merge two flowfiles together using MergeContent. The content of both files are JSON objects which I would like to append to each other and wrap both with a JSON key.
File 1:
"Create":[
{
"Action": "Create",
"Book": {
"Id": "1234"
}
},
{
"Action": "Create",
"Video": {
"Id": 3245
}
}
]
File 2:
"Update":[
{
"Action": "Update",
"Book": {
"Id": "5376"
}
},
{
"Action": "Update",
"Video": {
"Id": "8267"
}
}
]
I need the final output to look like so:
{
"Transaction": {
"Action": "Transaction",
"Type": {
"Create": [
{fill in stuff}
],
"Update": [
{fill in stuff}
]
}
}
}
What is happening is that my MergeContent processor is picking up more than one flow file out of the Update or Create input and it is merging 'like' flow files together, instead of merging the separate content together.
Screenshot of my MergeContent :
You can add a MergeContent processor with
Delimiter Strategy = Text
Header = {
Footer = }
Demarcator = ,
in order to generate the core part
"Create": [
{fill in stuff}
],
"Update": [
{fill in stuff}
]
then apply a JSONJoltTransform with specification
[
{
"operation": "shift",
"spec": {
"#Transaction": "&.Action",
"#": "Transaction.Type"
}
}
]
in order to convert them to a formatted JSON value as desired such as
Related
I have this JSON
{
"srv_config": [{
"name": "db1",
"servers": ["srv1", "srv2"],
"prop": [{"source":"aa"},"destination":"bb"},{"source":"cc"},"destination":"cc"},]
}, {
"name": "db2",
"servers": ["srv2", "srv2"],
"prop": [{"source":"dd"},"destination":"dd"},{"source":"ee"},"destination":"ee"},]
}
]
}
I try to build a JMESPath expression to select the prop application in each object in the main array, but based on the existence of a string in the servers element.
To select all props, I can do:
*.props [*]
But how do I add condition that says "select only if srv1 is in servers list"?
You can use the contains function in order to filter based on a array containing something.
Given the query:
*[?contains(servers, `srv1`)].prop | [][]
This gives us:
[
{
"source": "aa",
"destination": "bb"
},
{
"source": "cc",
"destination": "cc"
}
]
Please mind that I am also using a bit of flattening here.
All this run towards a corrected version of you JSON:
{
"srv_config":[
{
"name":"db1",
"servers":[
"srv1",
"srv2"
],
"prop":[
{
"source":"aa",
"destination":"bb"
},
{
"source":"cc",
"destination":"cc"
}
]
},
{
"name":"db2",
"servers":[
"srv2",
"srv2"
],
"prop":[
{
"source":"dd",
"destination":"dd"
},
{
"source":"ee",
"destination":"ee"
}
]
}
]
}
I have the following (simplified) jOOQ query:
val result = context.select(
jsonObject(
key("id").value(ITEM.ID),
key("title").value(ITEM.NAAM),
key("resources").value(
jsonArrayAgg(ITEM_INHOUD.RESOURCE_ID).absentOnNull()
)
)
).from(ITEM).fetch()
Now the output that I want is:
[
{
"id": "0da04cc5-f70c-4fb3-b5c7-dc645d342631",
"title": "Title1",
"resources": [
"8b0f6d5c-67fc-47ca-be77-d1735e7721ce",
"ea0316db-1cfd-46d7-8260-5c1a4e65a0cd"
]
},
{
"id": "0f7e67e6-5187-47e2-9f1d-dab08feba38b",
"title": "Title2"
}
]
result.formtJSON() gives the following output:
{
"fields": [
{
"name": "json_object",
"type": "JSON"
}
],
"records": [
[
{
"id": "0da04cc5-f70c-4fb3-b5c7-dc645d342631",
"title": "Title 1"
}
]
]
}
Disabling the headers with result.formatJSON(JSONFormat.DEFAULT_FOR_RECORDS) will get me:
[
[
{
"id": "0da04cc5-f70c-4fb3-b5c7-dc645d342631",
"title": "Title1",
"resources": [
"8b0f6d5c-67fc-47ca-be77-d1735e7721ce",
"ea0316db-1cfd-46d7-8260-5c1a4e65a0cd"
]
}
],
[
{
"id": "0f7e67e6-5187-47e2-9f1d-dab08feba38b",
"title": "Title2"
}
]
]
where I don't want the extra array.
Further customizing the JSONformatter with result.formatJSON(JSONFormat().header(false).recordFormat(JSONFormat.RecordFormat.OBJECT)) I get:
[
{
"json_object": {
"id": "0da04cc5-f70c-4fb3-b5c7-dc645d342631",
"title": "Title1",
"resources": [
"8b0f6d5c-67fc-47ca-be77-d1735e7721ce",
"ea0316db-1cfd-46d7-8260-5c1a4e65a0cd"
]
}
},
{
"json_object": {
"id": "0f7e67e6-5187-47e2-9f1d-dab08feba38b",
"title": "Title2"
}
}
]
where I don't want the object wrapped in json_object.
Is there a way to get the output I want?
Doing it with Result.formatJSON()
This is clearly a flaw in the jOOQ 3.14.0 implementation of Result.formatJSON(). In the special case where there is only one column, and that column is of type JSON or JSONB, the column name may not really matter, and thus its contents should be flattened into the object describing the row. I've created a feature request for this: https://github.com/jOOQ/jOOQ/issues/10953. It will be available in jOOQ 3.15.0 and 3.14.4. You will be able to do this:
result.formatJSON(JSONFormat().header(false).wrapSingleColumnRecords(false));
The RecordFormat is irrelevant here. This works the same way for RecordFormat.ARRAY and RecordFormat.OBJECT
Doing it directly with SQL
Of course, you can always work around this by moving all the logic into SQL. You probably simplified your query by omitting a JOIN and GROUP BY. I'm assuming this is equivalent to what you want:
JSON result = context.select(
jsonArrayAgg(jsonObject(
key("id").value(ITEM.ID),
key("title").value(ITEM.NAAM),
key("resources").value(
select(jsonArrayAgg(ITEM_INHOUD.RESOURCE_ID).absentOnNull())
.from(ITEM_INHOUD)
.where(ITEM_INHOUD.ITEM_ID.eq(ITEM.ID))
)
))
).from(ITEM).fetchSingle().value1()
Note that JSON_ARRAYAGG() aggregates empty sets into NULL, not into an empty []. If that's a problem, use COALESCE()
I am trying to find out correct JOLT specification file for below:
Input file:
{
"network": "A",
"ips": {
"subnet1": "1.1.1.1",
"subnet2": "1.1.1.2",
"subnet3": "1.1.1.3",
"subnet4": "1.1.1.4"
}
}
Output file:
{
"allValues": [
{
"network": "A",
"ips": {
"subnet1": "1.1.1.1",
"subnet2": "1.1.1.2"
}
},
{
"network": "A",
"ips": {
"subnet3": "1.1.1.3",
"subnet4": "1.1.1.4"
}
}
]
}
Based on subnet values, I want to group the IPs(subnet1 and subnet2 in one) and (subnet3 and subnet4 in another), while all should have network value(A).
Can somebody please help.
If you know which subnets belong together (subnet1 and subnet2 for example), you can use something like the following:
[
{
"operation": "shift",
"spec": {
"ips": {
"subnet1": "allValues[0].ips.subnet1",
"subnet2": "allValues[0].ips.subnet2",
"subnet3": "allValues[1].ips.subnet3",
"subnet4": "allValues[1].ips.subnet4"
},
"network": "allValues[0].network",
"#(0,network)": "allValues[1].network"
}
}
]
I am implementing a restful service where I am getting the pdf names and their ids from the database in the JSON fromat. Which one of the both the convenient JSON resful service response?
First Option:
{
"results": {
"documentNames": [
"test.pdf",
"ireport-ultimate-guide.pdf",
"sending report.pdf",
"Motor Hour.pdf"
],
"documentds": [
21116,
21117,
21118,
21119
]
}
}
Second Option:
{
"results": {
"21116": "test.pdf",
"21117": "ireport-ultimate-guide.pdf",
"21118": "sending report.pdf",
"21119": "Motor Hour.pdf"
}
}
I would use this "third option": The result is a list of object.
{
"result": [{
"id": "21116",
"filename": "test.pdf"
},
{
"id": "21117",
"filename": "ireport-ultimate-guide.pdf"
},
{
"id": "21118",
"filename": "sending report.pdf"
},
{
"id": "21119",
"filename": "Motor Hour.pdf"
}
]
}
because it better models the object structure.
I would create an entity for each document that contains both name and I'd.
[
{"name": "doc_1", "id": 123},
{"name": "doc_2", "id": 456}
]
Hi I am new to JOLT transformation. I need to transform the input json using JOLT to get below seen output. Please help me in below transformation:
input:
{
"image": [
"content1",
"content2",
"content3"
],
"legal": [
"legal1",
"legal2",
"legal3"
],
"hyper": [
"hyper1",
"hyper2",
"hyper3"
]
}
output:
[
{
"image": "content1",
"legal": "legal1",
"hyper": "hyper1"
},
{
"image": "content1",
"legal": "legal1",
"hyper": "hyper1"
},
{
"image": "content1",
"legal": "legal1",
"hyper": "hyper1"
}
]
Spec
[
{
"operation": "shift",
"spec": {
"*": { // image, legal, etc
"*": { // array
"*": { // content1, legal1, etc
"$": "[&2].&3" // grab "content1" and use it as output
// send it to an output doc that is a top level array
// indexed by looking 3 levels up the tree [&2]
}
}
}
}
}
]