I am trying to fetch data from Hbase using GetHBase process and the result is as below
{
"row" : "r5",
"cells" : {
"cf1:ABC:" : "V1",
"cf1:DEF" : "V2",
"cf1:HIJ" : "V3",
"cf1:KLM" : "V4"
}
}
Now, I want to filter the json for specific columns
expected result
{"DEF":"V2","KLM":"V4"}
Flow tried
GetHbase --> evaluateJsonPath
Just use JoltTransformJSON, set Jolt Transformation DSL as Shift and set Jolt Specification as:
{
"cells": {
"cf1:DEF": "DEF",
"cf1:KLM": "KLM"
}
}
this will transform your input json into your desired json.
Jolt Transform is an open source library that helps you transform Json documents using other Json documents as specification.
If you'd like to parameterize them, you can do that using parameters using NiFi 1.10.0 like so:
{
"cells": {
"cf1:#{param1}": "#{param1}",
"cf1:#{param2}": "#{param2}"
}
}
Or using NiFi 1.4.0+ with expression language and the variable registry like so:
{
"cells": {
"cf1:${param1}": "${param1}",
"cf1:${param2}": "${param2}"
}
}
Everything is linked to a documentation so you can read about it all :)
Related
I want to store some information to the mongodb collection which has the following json format:
{
"Pods" : {
"podName" : {
"container_count" : 2,
"containers" : [
{
"container1Name" : "container1",
"container1Image" : "container1image"
},
{
"container2Name" : "container2",
"container2Image" : "container2image"
}
]
}
}
}
I am using golang to do this, But i am getting some syntax errors as this is not really the correct format. I already converted my JSON to MongoDB schema with an online tool, Please find my code below:
coll := mongoConnection.Database(mongo_db).Collection(collection)
// doc := bson.D{{"pod_name", pod_name}, {"containers", containers}}
doc := bson.D{{"pod_name", podname}, {"container_count", countainer_count}, "containers", containers}
But my problem is, how can I format the containers variable so it looks like JSON objects inside an array?
...
"containers": [
{
"container1Name": "container1",
"container1Image": "container1image"
},
{
"container2Name": "container2",
"container2Image": "container2image"
}
]
Can someone help me please? Thank you
I have the following json:
{
"content":
[
{
"id":1,
"userId":2,
"storeId":8,
"userFirstName":"Max",
"userLastName":"Mustermann",
"city":"Berlin",
"spendQuantity":100,
"paymentStatus":"UNPAID",
"userBalanceStatus":null,
"rateObject":
{
"identifier":23,
"id":"432",
"rate":"1.9345345",
"symbol":"USD",
"rank":2,
}
},
{
"id":2,
"userId":2,
"storeId":3,
"userFirstName":"Newman",
"userLastName":"Mustermann",
"city":"Berlin",
"spendQuantity":1000,
"paymentStatus":"UNPAID",
"userBalanceStatus":null,
"rateObject":
{
"identifier":3,
"id":"234",
"rate":"1.922222245345",
"symbol":"USD",
"rank":2,
}
},
{
"id":3,
"userId":2,
"storeId":3,
"userFirstName":"Newman",
"userLastName":"Mustermann",
"city":"Munich",
"spendQuantity":3000,
"paymentStatus":"UNPAID",
"userBalanceStatus":null,
"rateObject":
{
"identifier":2332,
"id":"234",
"rate":"3.234234",
"symbol":"USD",
"rank":2,
}
},
{
"id":4,
"userId":2,
"storeId":3,
"userFirstName":"Newman",
"userLastName":"Mustermann",
"city":"Essen",
"spendQuantity":4000,
"paymentStatus":"UNPAID",
"userBalanceStatus":null,
"rateObject":
{
"identifier":234,
"id":"234",
"rate":"333.234234",
"symbol":"USD",
"rank":2,
}
}
}
But I need to verify it partially - Only the fields in the nested jsons where city is Berlin or Essen, but without the rateObject (I don't need to verify this part). With other words I need to verify nested jsons with ids- 1,2,4 - all fields, without the information in rateObject.
Partial Answer and Suggestion:
We can apply the filter condition in the JSON Query to fetch the matched details.
For Example: To get the id of the mentioned city,
JSON Query:
$.content[?(#.city=="Berlin" || #.city=="Essen")].id
Output:
[
1,
2,
4
]
Similarly, you can assert all the required fields using the filter JSON Query.
JMeter JSON Extractor will provide only one value at a time.So, you can either add some logic to verify all the occurrences or multiple verification can be added by specifying the id index ( $.content[?(#.city=="Berlin" || #.city=="Essen")].id[0] --> It gives the first occurrence Id )
If you want to validate multiple fields,then you can write the customized script in Bean Shell Post Processor.(Refer the below link and you will get some idea)
Extracting JSON Response using Bean Shell Postprocessor
If you are using the Bean Shell Post Processor, then required java jar files should be placed in JMeter ClassPath( Folder: apache-jmeter-4.0\lib\ext)
I want to update an attribute in a JSON file with the value i get from other processor. Below is my Original JSON file.
{
"applicant": {
"applicant-id": null
"full-name": "Tyrion Lannister",
"mobile-number" : "8435739739",
"email-id" : "tyrionlannister_casterlyrock#gmail.com"
},
"product": {
"product-category" : "Credit Card",
"product-type" : "Super Value Card - Titanium"
}
}
Below is my EvaluavateJsonPath Config where I extracted the applicant-id attribute.
Below is my GenerateFlowFile processor which generate an id value.
Now I need the update the applicant-id attribute with the value (899872120) in the Original JSON as below.
{
"applicant": {
"applicant-id": 899872120
"full-name": "Tyrion Lannister",
"mobile-number" : "8435739739",
"email-id" : "tyrionlannister_casterlyrock#gmail.com"
},
"product": {
"product-category" : "Credit Card",
"product-type" : "Super Value Card - Titanium"
}
}
I tried to use MergeContent to merge the 2 flows and i can see the Applicant-Id attribute value in the flow file after the MergeContent processor. I tried using UpdateAttribue to update Applicant-Id but i'm not able get the update JSON record.
Below is my MergeContent Configuration.
Is there anything i'm missing?
As of NiFi 1.2.0, the JoltTransformJSON processor supports NiFi Expression Language, so if you have your id value in the attribute "Applicant-id", you can use it in a Default JOLT spec:
{
"applicant": {
"applicant-id": "${Applicant-id}"
}
}
That should transform your input JSON to your desired output JSON.
what is the best approach to capture from the following array?
i only need to capture the value of ANY 'beginDate', e.g: 2017-05-01T08:30:00 could be a valid one in below example
i need to make sure the 'beschikbaar' = TRUE for the date that i'm capturing
i tried using json path extractor with similar lines: $..[?(#.beschikbaar == 'true')].beginDate but i'm facing syntax errors that i cant fix due to my limited regex/json path knowledge
the example array is;
{
"data":
[
[
{
"beginDate":"2017-05-01T08:00:00",
"eindDate":null,
"beschikbaar":false
},
{
"beginDate":"2017-05-01T08:15:00",
"eindDate":null,
"beschikbaar":false
},
{
"beginDate":"2017-05-01T08:30:00",
"eindDate":"2017-05-01T10:30:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T08:45:00",
"eindDate":"2017-05-01T10:45:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T09:00:00",
"eindDate":"2017-05-01T11:00:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T09:15:00",
"eindDate":"2017-05-01T11:15:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T09:30:00",
"eindDate":"2017-05-01T11:30:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T09:45:00",
"eindDate":"2017-05-01T11:45:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T10:00:00",
"eindDate":"2017-05-01T12:00:00+02:00",
"beschikbaar":true
},
Don't use regular expressions for JSON data, JMeter provides JSON Extractor designed to work with JSON data via JSON Path Language so you should be able to get your "beginDate" with the query like:
$..[?(#.beschikbaar == true)].beginDate
Demo:
Check out JMeter's JSON Path Extractor Plugin - Advanced Usage Scenarios article for more detailed explanation and few more examples.
You can try this
(?s)\{.*?\"beginDate\":\"([^{]*?)\"[^{]+\"beschikbaar\":true.*?\}
(?s) is single-line modifier which makes . match the line break
You can test it at http://www.regexplanet.com/advanced/java/index.html
And set Template to $1$ means using the first group
I'm having trouble properly formatting one particular soap parameter using the node-soap module for node.js as a client, to a 3rd-party SOAP service.
The client.describe() for this method says this particular input should be in the shape of:
params: { 'param[]': {} }
I have tried a bunch of different JSON notations to try to fit my data to that shape.
Examples of formats that do NOT work:
"params": { "param": [ {"myParameterName": "myParameterValue"} ] }
"params": [ "param": { "name": "myParameterName", "_": "myParameterValue"} ]
"params": { "param" : [ {"name": "myParameterName", "_": "myParameterValue"} ] }
"params": { "param[]": {"myParameterName": "myParameterValue" } }
"params": { "param[myParameterName]": {"_": "myParameterValue" } }
I must be overlooking something, and I suspect I'm going to feel like Captain Obvious when some nice person points out what I'm doing wrong.
Here is what DOES work, using other soap clients, and how they handle the "named parameter with a value"
soapUI for this method successfully accepts this particular input via XML in the shape of:
<ns:params>
<ns:param name="myParameterName">myParameterValue</ns:param>
</ns:params>
Also, using PHP, I can successfully make the call by creating a stdClass of arrays like so:
$parms = new stdClass;
$parms->param = array(
array(
"name"=>"myParameterName","_"=>"myParameterValue"
)
);
and then eventually passing
'params' => $parms
to the PHP soap client
Many thanks!
To get a better look at what XML was being generated by node-soap, I added a console.log(message) statement to the node_modules/soap/lib/client.js after the object-to-XML encoding. I then began experimenting with various JSON structures to figure out empirically how they were mapping to XML structures.
I found a JSON structure for node-soap to generate the XML in my 3rd-party's required named-parameter-with-value format. I was completely unaware of the "$value" special keyword. Looks like this may have been added in the 0.4.6 release from mid-June 2014. See the change history
"params": [
{
"param": {
"attributes": {
"name": "myParameterName"
},
$value: "myParameterValue"
}
}
]
(note the outer array, which gives me the luxury of specifying multiple "param" entries, which is sometimes needed by this particular 3rd-party API)
generates this XML:
<tns:params>
<tns:param name="myParameterName">myParameterValue</tns:param>
</tns:params>
which perfectly matches the structure in soapUI (which I already knew worked) of:
<ns:params>
<ns:param name="myParameterName">myParameterValue</ns:param>
</ns:params>