I have the following structure in my mondoDB:
"source" : [
{
"source_id" : "800000021",
"source" : "Test1"
}
]
"state" : "AM",
"owner_phone" : "0",
"latitude" : 0,
"leedon_activated" : "True",
"zip_code" : "12345",
"fax" : "0",
I want to add another json to the source array for the following resut:
"source" : [
{
"source_id" : "800000021",
"source" : "Test1"
},
{
"source_id" : "800000022",
"source" : "Test2"
}
]
"state" : "AM",
"owner_phone" : "0",
"latitude" : 0,
"leedon_activated" : "True",
"zip_code" : "12345",
"fax" : "0"
What will be MongoDB query for it?
Read about $push here
Your code will something like this
yourmodelname.update({ /* some conditions to updating files */ },{$push: {"source": {"source_id" : "213123", "source" : "asdasd"}}})
Related
how can I have transformed my json
{
"clients": [
{
"id" : "qwerty",
"accounts" : [{"number" : "6666"}, {"number" : "7777"}]
},
{
"id" : "zxcvb",
"accounts" : [{"number" : "1111"}, {"number" : "2222"}]
}
]
}
into following type of json? using JQ
{
"items": [
{
"id" : "qwerty",
"number" : "6666"
},{
"id" : "qwerty",
"number" : "7777"
},{
"id" : "zxcvb",
"number" : "1111"
},{
"id" : "zxcvb",
"number" : "2222"
}]
}
What kind of tools from JQ can help me? I can't choose any possible way to do it
Something like this should do the trick:
{items: [.clients[] | {id} + .accounts[]]}
Online demo
Hello can someone help me extract the value of user parameter which is "testuser1"
I tried to use this JSON Path expression $..data I was able to extract the entire response but unable to extract user parameter. Thanks in advance
{
"data": "{ "took" : 13, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 1.0, "hits" : [ { "_index" : "bushidodb_history_network_eval_ea9656ef-0a9b-474b-8026-2f83e2eb9df1_2021-april-10", "_type" : "network", "_id" : "6e2e58be-0ccf-3fb4-8239-1d4f2af322e21618059082000", "_score" : 1.0, "_source" : { "misMatches" : [ "protocol", "state", "command" ], "instance" : "e3032804-4b6d-3735-ac22-c827950395b4|0.0.0.0|10.179.155.155|53|UDP", "protocol" : "UDP", "localAddress" : "0.0.0.0", "localPort" : "12345", "foreignAddress" : "10.179.155.155", "foreignPort" : "53", "command" : "ping yahoo.com ", "user" : "testuser1", "pid" : "10060", "state" : "OUTGOINGFQ", "rate" : 216.0, "originalLocalAddress" : "192.168.100.229", "exe" : "/bin/ping", "md5" : "f9ad63ce8592af407a7be43b7d5de075", "dir" : "", "agentId" : "abcd-dcd123", "year" : "2021", "month" : "APRIL", "day" : "10", "hour" : "12", "time" : "1618059082000", "isMerged" : false, "timestamp" : "Apr 10, 2021 12:51:22 PM", "metricKey" : "6e2e58be-0ccf-3fb4-8239-1d4f2af322e2", "isCompliant" : false }, "sort" : [ 1618059082000 ] } ] }, "aggregations" : { "count_over_time" : { "buckets" : [ { "key_as_string" : "2021-04-10T08:00:00.000-0400", "key" : 1618056000000, "doc_count" : 1 } ] } }}",
"success": true,
"message": {
"code": "S",
"message": "Get Eval results Count Success"
}
}
Actual Response:
Images
What you posted doesn't look like a valid JSON to me.
If in reality you're getting what's in your image, to wit:
{
"data": "{ \"took\" : 13, \"timed_out\" : false, \"_shards\" : { \"total\" : 5, \"successful\" : 5, \"skipped\" : 0, \"failed\" : 0 }, \"hits\" : { \"total\" : 1, \"max_score\" : 1.0, \"hits\" : [ { \"_index\" : \"bushidodb_history_network_eval_ea9656ef-0a9b-474b-8026-2f83e2eb9df1_2021-april-10\", \"_type\" : \"network\", \"_id\" : \"6e2e58be-0ccf-3fb4-8239-1d4f2af322e21618059082000\", \"_score\" : 1.0, \"_source\" : { \"misMatches\" : [ \"protocol\", \"state\", \"command\" ], \"instance\" : \"e3032804-4b6d-3735-ac22-c827950395b4|0.0.0.0|10.179.155.155|53|UDP\", \"protocol\" : \"UDP\", \"localAddress\" : \"0.0.0.0\", \"localPort\" : \"12345\", \"foreignAddress\" : \"10.179.155.155\", \"foreignPort\" : \"53\", \"command\" : \"pingyahoo.com\", \"user\" : \"testuser1\", \"pid\" : \"10060\", \"state\" : \"OUTGOINGFQ\", \"rate\" : 216.0, \"originalLocalAddress\" : \"192.168.100.229\", \"exe\" : \"/bin/ping\", \"md5\" : \"f9ad63ce8592af407a7be43b7d5de075\", \"dir\" : \"\", \"agentId\" : \"abcd-dcd123\", \"year\" : \"2021\", \"month\" : \"APRIL\", \"day\" : \"10\", \"hour\" : \"12\", \"time\" : \"1618059082000\", \"isMerged\" : false, \"timestamp\" : \"Apr10, 202112: 51: 22PM\", \"metricKey\" : \"6e2e58be-0ccf-3fb4-8239-1d4f2af322e2\", \"isCompliant\" : false }, \"sort\" : [ 1618059082000 ] } ] }, \"aggregations\" : { \"count_over_time\" : { \"buckets\" : [ { \"key_as_string\" : \"2021-04-10T08: 00: 00.000-0400\", \"key\" : 1618056000000, \"doc_count\" : 1 } ] } }}",
"success": true,
"message": {
"code": "S",
"message": "Get Eval results Count Success"
}
}
the easiest way is just using 2 JSON Extractors:
Extract data attribute value into a JMeter Variable from the response
Extract user attribute value into a JMeter variable from ${data} JMeter Variable:
Demo:
If the response looks like exactly you posted you won't be able to use JSON Extractors and will have to treat it as normal text so your choice is limited to Regular Expression Extractor, example regular expression:
"user"\s*:\s*"(\w+)"
Add Regular Expression extractor with the corresponding request and extract it. Use the below expression.
Expression: "user" : "(.*?)"
Ref: https://jmeter.apache.org/usermanual/regular_expressions.html
Regular Expression Extractor Sample
I am new to elasticserach. Below is the Sample data on which elastic query needs to run. I am trying to get those docs in which account_type is "credit card" and source_name is 'SOMEVALUE'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "bureau_data",
"_type" : "_doc",
"_id" : "bda57e01-c564-4cdc-bb8d-79bd2db9d2f8",
"_score" : 1.0,
"_source" : {
"userid" : "bda57e01-c564-4cdc-bb8d-79bd2db9d2f8",
"raw_derived" : {
"gender" : "MALE",
"firstname" : "trsqlsz",
"middlename" : "rgj",
"lastname" : "ggksb",
"mobilephone" : "2125954664",
"dob" : "1988-06-28 00:00:00",
"applications" : [
{
"applicationid" : "c7fb0147-22fd-4a5e-8851-98241de6aa50",
"createdat" : "2019-06-07 19:28:54",
"updatedat" : "2019-06-07 19:28:55",
"source" : "4",
"source_name" : "EXPERIAN",
"applicationcreditreportid" : "b67f9180-9bb6-485c-9cfc-e7ccf9a70a69",
"accounts" : [
{
"applicationcreditreportaccountid" : "c5de28c4-cac9-4390-852a-96f143cb0b62",
"currentbalance" : 418288,
"institutionid" : "021d58b4-aba5-42c9-8d39-304a78d34aea",
"accounttypeid" : "5",
"institution_name" : "HDFC BANK",
"account_type_name" : "Personal Loan"
}
]
}
]
}
}
}
I have tried the below query and its working fine. I need if we have any optimized way to query the multiple nested fields
GET /my_index/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "raw_derived.applications.accounts",
"query": {
"bool": {
"must": [
{"match": {
"raw_derived.applications.accounts.account_type_name": "Credit Card"
}}
]
}
}
}
},
{
"nested": {
"path": "raw_derived.applications",
"query": {
"bool": {
"must": [
{"match": {
"raw_derived.applications.source_name": "CIBIL"
}}
]
}
}
}
}
]
}
}
}
If I will query on the multiple nested fields it will become very long Please suggest any other way to query nested fields or multiple AND
Well your optimizations should always start with your data model / mapping since it's mostly the cause of performance issues and not your queries.
That being said, you can avoid the nested query by flattening your data. A flattened data model would lead to one document per application and account element.
Since elasticsearch is a non-relational data store, it is completely fine to index "redundant" data. This is not a lazy appraoch but a common way to handle these type of data structures.
Sample document #1:
{
"_index" : "bureau_data",
"_type" : "_doc",
"_id" : "bda57e01-c564-4cdc-bb8d-79bd2db9d2f8",
"_score" : 1.0,
"_source" : {
"userid" : "bda57e01-c564-4cdc-bb8d-79bd2db9d2f8",
"gender" : "MALE",
"firstname" : "trsqlsz",
"middlename" : "rgj",
"lastname" : "ggksb",
"mobilephone" : "2125954664",
"dob" : "1988-06-28 00:00:00",
"applicationid" : "c7fb0147-22fd-4a5e-8851-98241de6aa50",
"createdat" : "2019-06-07 19:28:54",
"updatedat" : "2019-06-07 19:28:55",
"source" : "4",
"source_name" : "EXPERIAN",
"applicationcreditreportid" : "b67f9180-9bb6-485c-9cfc-e7ccf9a70a69",
"applicationcreditreportaccountid" : "c5de28c4-cac9-4390-852a-96f143cb0b62",
"currentbalance" : 418288,
"institutionid" : "021d58b4-aba5-42c9-8d39-304a78d34aea",
"accounttypeid" : "5",
"institution_name" : "HDFC BANK",
"account_type_name" : "Personal Loan"
}
}
If the same user creates another account you would send the very same ("redundant") data, except for that other account element/data like so:
{
"_index" : "bureau_data",
"_type" : "_doc",
"_id" : "another, from es generated id",
"_score" : 1.0,
"_source" : {
"userid" : "bda57e01-c564-4cdc-bb8d-79bd2db9d2f8",
"gender" : "MALE",
"firstname" : "trsqlsz",
"middlename" : "rgj",
"lastname" : "ggksb",
"mobilephone" : "2125954664",
"dob" : "1988-06-28 00:00:00",
"applicationid" : "c7fb0147-22fd-4a5e-8851-98241de6aa50",
"createdat" : "2019-06-07 19:28:54",
"updatedat" : "2019-06-07 19:28:55",
"source" : "4",
"source_name" : "EXPERIAN",
"applicationcreditreportid" : "b67f9180-9bb6-485c-9cfc-e7ccf9a70a69",
"applicationcreditreportaccountid" : "the new id",
"currentbalance" : 4711,
"institutionid" : "foo",
"accounttypeid" : "bar",
"institution_name" : "foo bar",
"account_type_name" : "foo baz"
}
}
With that kind of data model, you can run simple queries to get your results:
GET /my_index/_search
{
"query": {
"bool": {
"must": [
{
"match":{
"account_type_name": "Credit Card"
}
},
{
"match":{
"source_name": "CIBIL"
}
}
]
}
}
}
How can I push new key and value in JSON array?
I tried I used push keyword in update query but I got a different output. I used:
db.users.updateOne({"name":"viki"},{$push{"address.district":"thambaram"}})
I have this document:
{ "_id" : ObjectId("58934f10c7592b1494fd9a4d"), "name" : "viki", "age" : 100, "subject" : [ "c", "node.js", "java" ], "address" : { "city" : "chennai", "state" : "tamilnadu", "pincode" : "123" } }
I want to add "district":"thambaram" in address json array
I need like:
{ "_id" : ObjectId("58934f10c7592b1494fd9a4d"), "name" : "viki", "age" : 100, "subject" : [ "c", "node.js", "java" ], "address" : { "city" : "chennai", "state" : "tamilnadu", "pincode" : "123","district":"thambaram"} }
Use $set
db.users.updateOne({"name":"viki"},{$set:{"address.district":"thambaram"}})
This should work.
The $push operator appends a specified value to an array. In your case you should use $set
I've got a MongoDB which I query and the result I serialize and this string I send to my ftl template. Below is the serialized result:
[
{
"id" : "10",
"title" : "Test Title 1",
"partner" : {
"id" : "1",
"name" : "partner 1 ",
"location" : [{
"locationname" : "locationname 1a",
"city" : ""
},{
"locationname" : "locationname 1b",
"city" : ""
}]
}
},
{
"id" : "6",
"title" : "Test Title 2",
"partner" : {
"id" : "1",
"name" : "partner 2 ",
"location" : [{
"locationname" : "locationname 2b",
"city" : ""
}]
}
}
]
How would I use this in my ftl template?
Thanks for any help.
If you really have to serialize before giving the result to FreeMarker... The JSON syntax for maps and lists happens to be a subset of FTL, so assuming the serialized result is in res, res?eval will give you a list of maps.