I want to get the productLines for the prodType ULTRA for the Json array below. I get the array of maps and uses findIndexValues to get the index but
it doesn't work. What am I missing? I looked at similar examples which are less complex in structure and did not see much difference from what I'm attempting
Here is my data:
def static modelData="""
{
"models": [
{
"transactionId": "01-PROD0021",
"prodCode": "ISN-2017WDE",
"product": "VASCULAR DNNT",
"prodType": "SDISCNT",
"productLines": [
{
"productLineId": "ELECT-2221",
"productDescriptor": "XTRA-SONIC DNNP",
"unitPrice": "",
},
{
"productLineId": "ELECT-2223",
"productDescriptor": "HEADPH",
"unitPrice": "1.33",
}
]
},
{
"transactionId": "01-PROD0024",
"prodCode": "ISN-5543XDR",
"product": "ULTRASOUND DEEP SONAR",
"prodType": "ULTRA",
"productLines": [
{
"productLineId": "MEDCN-XTR221",
"productDescriptor": "ELECTRONIC RESPR",
"unitPrice": "2.44",
},
{
"productLineId": "MEDCN-XTR376",
"productDescriptor": "SPNG ELECTRONIC DEFIB",
"unitPrice": "6.22",
}
}
]
]
}
"""
Here is my attempt:
def parsed = new JsonSlurper().parseText(modelData)
// Find index of the prodCode with 'ULTRA'
int [] vals=parsed.data.findIndexValues{
it -> it.key=='prodType' && it.value=='ULTRA'}
//Does not print anything
vals?.each {println "Found an index! ${it}" }
There's a couple of things wrong with the code:
1. You loop through a "data" node, where there is none. Use parsed.data
2. The each node is a map-like structure. Hence you check if the map contains a key prodType with value ULTRA. Use it.prodType == 'ULTRA'.
Pro-tip:
1. You can print what the data looks like on closures so that you will get to the solution quicker.
I was able to figure it out
def parsed = new JsonSlurper().parseText(modelData)
def vals = parsed.models.find{ it.prodType == 'ULTRA' }?.productLines
Related
I have have a JSON file where I have IDs with tasks. Some tasks can be empty. I want to put the ID into the tasks where tasks are not empty.
[
{
"id": 1961126,
"tasks": [
{
"id": 70340700,
"title": "Test1",
},
{
"id": 69801130,
"title": "Test15A",
}
]
},
{
"id": 1961126,
"tasks": []
}
]
I would like to get the tasks array updated to look like
[
{
"id": 1961126,
"tasks": [
{
**"sId":1961126,**
"id": 70340700,
"title": "Test1",
},
{
**"sId":1961126,**
"id": 69801130,
"title": "Test15A",
}
]
},
{
"id": 1961126,
"tasks": []
}
]
I can't figure out how to get the id from the object into the nested array. Here is what I have come up with
jq 'map(.tasks[0]|select( . != null )|.sId = .id)' file.json
This is only pulling in the same id. I have tired to put in [].id but I get a error Cannot index number with string "id". I am still learning how to deal with nested arrays and objects.
Save the ID in a variable and add it as a new field to each array member.
jq 'map(.id as $sId | .tasks[] += {$sId})' file.json
Demo
Note #1: Get rid of the final , within each object (see the Demo), as it's not proper JSON.
Note #2: Object fields generally have no order, but if you want to have the propagated ID shown first, as seen in your expected output, you could try to replace += {$sId} (which by itself is shorthand for |= . + {$sId}) with |= {$sId} + . to flip the order of generation (Demo). Although there is no guarantee that it stays that way with further processing.
I have a Datastream[ObjectNode] which I read as deserialized json from a kafka topic. One of the element of this ObjectNode is actually an array of events. This array has varying length. The incoming json stream looks like this :
{
"eventType": "Impression",
"deviceId": "359849094258487",
"payload": {
"vertical_name": "",
"promo_layout_type": "aa",
"Customer_Id": "1011851",
"ecommerce": {
"promoView": {
"promotions": [{
"name": "/-category_icons_all",
"id": "300275",
"position": "slot_5_1",
"creative": "Central/Gift Card/00000001B890D1739913DDA956AB5C79775991EC"
}, {
"name": "/-category_icons_all",
"id": "300276",
"position": "slot_6_1",
"creative": "Lifestyle/Gift Card/00000001B890D1739913DDA956AB5C79775991EC"
}, {
"name": "/-category_icons_all",
"id": "413002",
"position": "slot_7_1",
"creative": "Uber/Deals/00000001B890D1739913DDA956AB5C79775991EC"
}]
}
}
}
}
I want to be able to explode the promotions array so that each element inside becomes an individual message which can be written to a sink kafka topic. Does Flink provide the explode feature in DataStream and/or Table API?
I have tried to do a RichFlatMap on this stream to be able to collect individual rows but this also just returns me a DataStream[Seq[GenericRecord]] as below:
class PromoMapper(schema: Schema) extends RichFlatMapFunction[node.ObjectNode,Seq[GenericRecord]] {
override def flatMap(value: ObjectNode, out: Collector[Seq[GenericRecord]]): Unit = {
val promos = value.get("payload").get("ecommerce").get("promoView").get("promotions").asInstanceOf[Seq[node.ObjectNode]]
val record = for{promo <- promos} yield {
val processedRecord: GenericData.Record = new GenericData.Record(schema)
promo.fieldNames().asScala.foreach(f => processedRecord.put(f,promo.get(f)))
processedRecord
}
out.collect(record)
}
}
Please help.
Using a flatmap is the right idea (not sure why you bothered with a RichFlatMap, but that's a detail).
Seems like you should be calling out.collect(processedRecord) for each element inside the for loop, rather than once on the Seq produced by that loop.
I have this json code:
{
"kind": "something",
"rules": [
{
"rule1": [
"something1",
"something2",
"something3"
],
"rule2": "something"
}
]
}
I want to add something4 to rule1. How can I do this in Ruby?
thanks
Let json_string be your JSON string
require 'json'
h = JSON.parse(json_string)
h["rules"][0]["rule1"] << "something4"
Your Nesting Structure Uses Keys and Indexes
Your problem is that your JSON is really poorly-designed for finding specific keys, because the various rules are actually wrapped inside an array. In your example, you only have one, but if you have more array elements then you've got to find the right array element to modify. It can be done with a lot of work via Enumerable#each_with_index or a related method, but if you are confident of the structure of your JSON you can take a shortcut.
require 'json'
json = <<~'EOF'
{
"kind": "something",
"rules": [
{
"rule1": [
"something1",
"something2",
"something3"
],
"rule2": "something"
}
]
}
EOF
# Get the first array from "rules", and append the string
# to the array inside the hash value referenced by the
# "rule1" key.
hash['rules'].first['rule1'] << 'something4'
puts JSON.pretty_generate(hash)
{
"kind": "something",
"rules": [
{
"rule1": [
"something1",
"something2",
"something3",
"something4"
],
"rule2": "something"
}
]
}
In the real world, you'd likely have to iterate over the rules array to find the index of the array element you want to modify. After that, you can find the rule1 key and append the value you want to the value's hash.
Saying that you assign that code to a variable
settings = {
"kind": "something",
"rules": [
{
"rule1": [
"something1",
"something2",
"something3"
],
"rule2": "something"
}
]
}
you can do this:
settings["rules"][0]["rule1"] << "test"
I pushed a json file ( as shown below ) to ES using the following code :
with open('test.json','rb') as payload:
headers = {'content-type': 'application/json'}
r = requests.post('http://localhost:9200/test_nest_json/1',data=payload, verify=False, headers=headers)
{
"data": [
{
"keyword": "abc",
"lists": [
{
"item_val": "some_val"
}
],
"another_key": "some_key"
},
{
"keyword": "xyz",
"lists": [
{
"item_val":"another_val"
}
],
"another_key": "pqr"
}
]
}
I tried updating the mappings and used the term query but still it results in displaying all the indices. I am not able to query only one keyword like "data.keyword" = "abc" using term query.
Looks like you are having a problem with nested object
https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html
The reason for this cross-object matching, as discussed in Arrays of
Inner Objects, is that our beautifully structured JSON document is
flattened into a simple key-value format in the index
So the effective document stored looks like this:
{
"data.keyword": [ abc, xyz ],
"data.another_key": [ some_key, pqr ],
}
Which means the query you posted will match any document, as long as at least one of the nested object contains the xyz keyword. I recommend reading the link above for clarification.
This is what worked for me :
es.indices.refresh(index="test-index")
with open('abc.json','rb') as payload:
json_data = json.load(payload);
leng = len(json_data["data"])
for i in range (leng):
doc = json.dumps(json_data["data"][i]);
res = es.index(index="sample-index", doc_type='pdf',id=str(uuid.uuid4()), body=doc)
I am parsing the json and extracting the array items one by one and push it to ElasticSearch.
{
"keyword": "abc",
"lists": [
{
"item_val": "some_val"
}
],
"another_key": "some_key"
},
Still looking for an optimised solution.
I am new to Ember and JSON. I want to parse a JSON object that is below with typeahead library
and access nested object values by searching their keys.
I have this Json format:
return [
{
"id": 1,
"category_name": "Supermarket",
"category_description": "SUPER MARKET",
"image_url": "",
"merchants": [
{
"name": "CARREFOUR",
"id": 12,
"merchant_type_id": 1,
"merchant_type_description": "Gold",
"merchant_redeption_rate": 0.002500,
"image_url": "https://jpg",
"branches": [
{
"id": 123456,
"latitude": 37.939483,
"area": "ΑΓ. ΔΗΜΗΤΡΙΟΣ",
"zip": "12345"
},
{
"id": 4567890,
"longitude": 23.650622,
"area": "ΑΓ. ΙΩΑΝΝΗΣ ΡΕΝΤΗΣ",
"zip": "12345"
}
]
},
{
"name": "CAFCO",
"id": 13,
"merchant_type_id": 3,
"merchant_type_description": "None",
"merchant_redeption_rate": 0.002500,
"image_url": "https:.jpg",
"branches": [
{
"id": 127890,
"latitude": 38.027870,
"area": "ΠΕΡΙΣΤΕΡΙ",
"zip": "12345"
}
]
}
]
},
{
"id": 2,
"category_name": "Πολυκαταστήματα",
"category_description": "ΠΟΛΥΚΑΤΑΣΤΗΜΑ",
"image_url": "",
"merchants": [
{
"name": "AGGELOPOYLOS CHR.",
"id": 15,
"merchant_type_id": 2,
"merchant_type_description": "Silver",
"merchant_redeption_rate": 0.002500,
"image_url": "https://www.nbg.gr/greek/retail/cards/reward-programmes/gonational/PublishingImages/aggelopoulos.jpg",
"branches": [
{
"id": 234780,
"latitude": 35.366118,
"longitude": 24.479461,
"address": "ΕΘΝ. ΜΑΚΑΡΙΟΥ 9 & ΕΛ. ΒΕΝΙΖΕΛΟΥ 1",
"area": "Ν. ΦΑΛΗΡΟ",
"zip": "12345"
}
]
}
]
}
];
--------------------------Updated----------------------------
For example, i want to search using typeahead the name of merchants and when the letter we write to search matches the name of merchants it will appear the corresponding category_name and backwards.
Example -> when i keyboard the s it will appear :
Category : Supermarket,
Name: CARREFOUR
Name: CAFCO
And the same output on the dropdown of search when i keyboard the letter c.
Any help?
New Jsbin example
The simplest way (in my mind) to get this to work is to create a computed property that will contain an array of latitudes. But how do we get there?
To get to latitude, you need to go through array of merchants and then array of branches. Being that this will be across multiple elements, you are going to end up with "array of arrays" type data structure, which is annoying to deal with. So, to simplify this, we can create a simple flatten function as follows:
flatten: function(origArray){
var newArr = [];
origArray.forEach(function(el) {
el.forEach(function(eachEl){
newArr.push(eachEl);
});
});
return newArr;
},
In addition to our function above, Ember already provides us with many other useful functions that can be used on arrays (see here). One of those is mapBy(property) which transforms an array into another array only keeping the values of the property we specified.
So, to create a lats (for latitudes) property, we can just do this:
lats: function(){
var merchantsArr = this.get('model').mapBy('merchants');
merchantsArr = this.flatten(merchantsArr);
var branchesArr = merchantsArr.mapBy('branches');
branchesArr = this.flatten(branchesArr);
return branchesArr.mapBy("latitude").compact();
}.property('model')
Above, I am basically using mapBy, flatten (see above) and compact which
Returns a copy of the array with all null and undefined elements removed.
Once you have the lats property with all the necessary data, the rest is easy.
Your call to component becomes:
{{x-typeahead data=lats name='category_name' selection=myColor}}
Note lats instead of model you originally were passing into the component.
And now, to access the value of data property in the component, you do
`this.get('data')`
which you can just pass in as the source like so:
source: substringMatcher(self.get('data'))
Working solution here
Update
Updating my answer based on your updated question.
OK, so this is getting a little more complicated. You now need more than just one property (latitude) from the object. You need category_name and merchant name.
In addition to mapBy, which just grabs one property out of array, Ember also has map which lets you transform the array into pretty much anything you want to:
lats: function(){
var merchantsArr = this.get('model').map(function(thing){
var category_name = thing.category_name;
return thing.merchants.map(function(merchant){
return {
"name": merchant.name,
"category": category_name
};
});
});
merchantsArr = this.flatten(merchantsArr);
return merchantsArr;
}.property('model')
The code above looks complicated, but it's basically just returning an array of top level objects' merchants accompanied by category_name. Since this is an array of arrays, we will need to flatten it.
Then, inside the component, we need to keep in mind that we are not just passing in an array of strings, but rather we are passing in an array of objects. Therefore, we need to look through object's properties (name and category) for a match
$.each(strs, function(i, str) {
if (substrRegex.test(str.name) || substrRegex.test(str.category)) {
matches.push(str);
}
});
Lastly, to actually display both category and merchant name, you need to tell Typeahead how to do that:
templates: {
suggestion: Handlebars.compile('<p>{{name}} – {{category}}</p>')
}
Working solution here