I'm using Factual api to fetch location data. Their restful service return data in JSON format as follow, but they are not using "usual" JSON format. There's no attribute key, instead, there's a “fields” that explains all the field keys.
So the question is how to retrieve the attribute I need? Please give an example if possible. Thanks in advance.
{
"response": {
"total_rows": 2,
"data": [
[
"ZPQAB5GAPEHQHDy5vrJKXZZYQ-A",
"046b39ea-0951-4add-be40-5d32b7037214",
"Hanko Sushi Iso Omena",
60.16216,
24.73907
],
[
"2TptHCm_406h45y0-8_pJJXaEYA",
"27dcc2b5-81d1-4a72-b67e-2f28b07b9285",
"Masabi Sushi Oy",
60.21707,
24.81192
]
],
"fields": [
"subject_key",
"factual_id",
"name",
"latitude",
"longitude"
],
"rows": 2,
"cache-state": "CACHED",
"big-data": true,
"subject_columns": [
1
]
},
"version": "2",
"status": "ok"
}
If you know the field name, and the data isn't guaranteed to stay in the same order, I would do a transform on the data so I can reference the fields by name:
var fieldIndex = {}
for (key in x.response.fields)
{
fieldIndex[x.response.fields[key]] = key;
}
for (key in x.response.data)
{
alert(x.response.data[key][fieldIndex.name]);
}
// Field map
var _subject_key = 0,
_factual_id = 1,
_name = 2,
_latitude = 3,
_longitude = 4;
// Example:
alert(_json.response.data[0][_factual_id]);
Demo: http://jsfiddle.net/AlienWebguy/9TEJJ/
I work at Factual. Just wanted to mention that we've launched the beta of version 3 of our API. Version 3 solves this problem directly, by including the attribute keys inline with the results, as you would hope. (Your question applies to version 2 of our API. If you're able to upgrade to version 3 you'll find some other nice improvements as well. ;-)
Related
I have a Symfony 3.2 project, and I need to filter data from a json column.
Given that we have an entity named "pack" with a json column named "settings" containing this kind of data:
{
"name": "My pack",
"blocks": [
{
"name": "Block 1",
"fields": [
{"label": "A", "value": "57"},
{"label": "B", "value": "100"}
]
},
{
"name": "Bock 2",
"fields": [
{"label": "C", "value": "80"}
]
}
]
}
I have to search packs with a field which has the label "B" and its value at "100", but each pack doesn't have same blocks and fields order.
So in my repository, using Doctrine\ORM\EntityRepository and opsway/doctrine-dbal-postgresql (for GET_JSON_FIELD and GET_JSON_OBJECT functions), this kind of condition works:
use Doctrine\ORM\EntityRepository;
class Packs extends EntityRepository
{
public function findFiltered(...)
{
return $this->createQueryBuilder('pack')
->andWhere("GET_JSON_FIELD(GET_JSON_OBJECT(pack.settings, '{blocks,0,fields,1}'), 'label') = :label")
->andWhere("GET_JSON_FIELD(GET_JSON_OBJECT(pack.settings, '{blocks,0,fields,1}'), 'value') = :value")
->setParameter('label', 'B')
->setParameter('value', '100')
;
}
}
But the problem is that I have to specify the precise block (the first block object), and the precise field (the second field object of the first block object). And my two condition aren't connected, it search if there is a label "B", then it search if there is a value "100". When I would like to have a research in all blocks and fields to find the good label for the good value. Any idea?
I found the good SQL request for my problem:
SELECT *
FROM pack p, json_array_elements(p.settings#>'{blocks}') blocks, json_array_elements(blocks#>'{fields}') fields
WHERE fields->>'label' = 'B' and fields->>'value' = '100';
But how I do that with doctrine?
Maybe this link can help you, it is a custom filter for a JSON type field, maybe it will serve as an example, but these functions with this bundle solved the problem for me. I hope this helps someone else too. Cheers!
I want to get the productLines for the prodType ULTRA for the Json array below. I get the array of maps and uses findIndexValues to get the index but
it doesn't work. What am I missing? I looked at similar examples which are less complex in structure and did not see much difference from what I'm attempting
Here is my data:
def static modelData="""
{
"models": [
{
"transactionId": "01-PROD0021",
"prodCode": "ISN-2017WDE",
"product": "VASCULAR DNNT",
"prodType": "SDISCNT",
"productLines": [
{
"productLineId": "ELECT-2221",
"productDescriptor": "XTRA-SONIC DNNP",
"unitPrice": "",
},
{
"productLineId": "ELECT-2223",
"productDescriptor": "HEADPH",
"unitPrice": "1.33",
}
]
},
{
"transactionId": "01-PROD0024",
"prodCode": "ISN-5543XDR",
"product": "ULTRASOUND DEEP SONAR",
"prodType": "ULTRA",
"productLines": [
{
"productLineId": "MEDCN-XTR221",
"productDescriptor": "ELECTRONIC RESPR",
"unitPrice": "2.44",
},
{
"productLineId": "MEDCN-XTR376",
"productDescriptor": "SPNG ELECTRONIC DEFIB",
"unitPrice": "6.22",
}
}
]
]
}
"""
Here is my attempt:
def parsed = new JsonSlurper().parseText(modelData)
// Find index of the prodCode with 'ULTRA'
int [] vals=parsed.data.findIndexValues{
it -> it.key=='prodType' && it.value=='ULTRA'}
//Does not print anything
vals?.each {println "Found an index! ${it}" }
There's a couple of things wrong with the code:
1. You loop through a "data" node, where there is none. Use parsed.data
2. The each node is a map-like structure. Hence you check if the map contains a key prodType with value ULTRA. Use it.prodType == 'ULTRA'.
Pro-tip:
1. You can print what the data looks like on closures so that you will get to the solution quicker.
I was able to figure it out
def parsed = new JsonSlurper().parseText(modelData)
def vals = parsed.models.find{ it.prodType == 'ULTRA' }?.productLines
I pushed a json file ( as shown below ) to ES using the following code :
with open('test.json','rb') as payload:
headers = {'content-type': 'application/json'}
r = requests.post('http://localhost:9200/test_nest_json/1',data=payload, verify=False, headers=headers)
{
"data": [
{
"keyword": "abc",
"lists": [
{
"item_val": "some_val"
}
],
"another_key": "some_key"
},
{
"keyword": "xyz",
"lists": [
{
"item_val":"another_val"
}
],
"another_key": "pqr"
}
]
}
I tried updating the mappings and used the term query but still it results in displaying all the indices. I am not able to query only one keyword like "data.keyword" = "abc" using term query.
Looks like you are having a problem with nested object
https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html
The reason for this cross-object matching, as discussed in Arrays of
Inner Objects, is that our beautifully structured JSON document is
flattened into a simple key-value format in the index
So the effective document stored looks like this:
{
"data.keyword": [ abc, xyz ],
"data.another_key": [ some_key, pqr ],
}
Which means the query you posted will match any document, as long as at least one of the nested object contains the xyz keyword. I recommend reading the link above for clarification.
This is what worked for me :
es.indices.refresh(index="test-index")
with open('abc.json','rb') as payload:
json_data = json.load(payload);
leng = len(json_data["data"])
for i in range (leng):
doc = json.dumps(json_data["data"][i]);
res = es.index(index="sample-index", doc_type='pdf',id=str(uuid.uuid4()), body=doc)
I am parsing the json and extracting the array items one by one and push it to ElasticSearch.
{
"keyword": "abc",
"lists": [
{
"item_val": "some_val"
}
],
"another_key": "some_key"
},
Still looking for an optimised solution.
I am new to Ember and JSON. I want to parse a JSON object that is below with typeahead library
and access nested object values by searching their keys.
I have this Json format:
return [
{
"id": 1,
"category_name": "Supermarket",
"category_description": "SUPER MARKET",
"image_url": "",
"merchants": [
{
"name": "CARREFOUR",
"id": 12,
"merchant_type_id": 1,
"merchant_type_description": "Gold",
"merchant_redeption_rate": 0.002500,
"image_url": "https://jpg",
"branches": [
{
"id": 123456,
"latitude": 37.939483,
"area": "ΑΓ. ΔΗΜΗΤΡΙΟΣ",
"zip": "12345"
},
{
"id": 4567890,
"longitude": 23.650622,
"area": "ΑΓ. ΙΩΑΝΝΗΣ ΡΕΝΤΗΣ",
"zip": "12345"
}
]
},
{
"name": "CAFCO",
"id": 13,
"merchant_type_id": 3,
"merchant_type_description": "None",
"merchant_redeption_rate": 0.002500,
"image_url": "https:.jpg",
"branches": [
{
"id": 127890,
"latitude": 38.027870,
"area": "ΠΕΡΙΣΤΕΡΙ",
"zip": "12345"
}
]
}
]
},
{
"id": 2,
"category_name": "Πολυκαταστήματα",
"category_description": "ΠΟΛΥΚΑΤΑΣΤΗΜΑ",
"image_url": "",
"merchants": [
{
"name": "AGGELOPOYLOS CHR.",
"id": 15,
"merchant_type_id": 2,
"merchant_type_description": "Silver",
"merchant_redeption_rate": 0.002500,
"image_url": "https://www.nbg.gr/greek/retail/cards/reward-programmes/gonational/PublishingImages/aggelopoulos.jpg",
"branches": [
{
"id": 234780,
"latitude": 35.366118,
"longitude": 24.479461,
"address": "ΕΘΝ. ΜΑΚΑΡΙΟΥ 9 & ΕΛ. ΒΕΝΙΖΕΛΟΥ 1",
"area": "Ν. ΦΑΛΗΡΟ",
"zip": "12345"
}
]
}
]
}
];
--------------------------Updated----------------------------
For example, i want to search using typeahead the name of merchants and when the letter we write to search matches the name of merchants it will appear the corresponding category_name and backwards.
Example -> when i keyboard the s it will appear :
Category : Supermarket,
Name: CARREFOUR
Name: CAFCO
And the same output on the dropdown of search when i keyboard the letter c.
Any help?
New Jsbin example
The simplest way (in my mind) to get this to work is to create a computed property that will contain an array of latitudes. But how do we get there?
To get to latitude, you need to go through array of merchants and then array of branches. Being that this will be across multiple elements, you are going to end up with "array of arrays" type data structure, which is annoying to deal with. So, to simplify this, we can create a simple flatten function as follows:
flatten: function(origArray){
var newArr = [];
origArray.forEach(function(el) {
el.forEach(function(eachEl){
newArr.push(eachEl);
});
});
return newArr;
},
In addition to our function above, Ember already provides us with many other useful functions that can be used on arrays (see here). One of those is mapBy(property) which transforms an array into another array only keeping the values of the property we specified.
So, to create a lats (for latitudes) property, we can just do this:
lats: function(){
var merchantsArr = this.get('model').mapBy('merchants');
merchantsArr = this.flatten(merchantsArr);
var branchesArr = merchantsArr.mapBy('branches');
branchesArr = this.flatten(branchesArr);
return branchesArr.mapBy("latitude").compact();
}.property('model')
Above, I am basically using mapBy, flatten (see above) and compact which
Returns a copy of the array with all null and undefined elements removed.
Once you have the lats property with all the necessary data, the rest is easy.
Your call to component becomes:
{{x-typeahead data=lats name='category_name' selection=myColor}}
Note lats instead of model you originally were passing into the component.
And now, to access the value of data property in the component, you do
`this.get('data')`
which you can just pass in as the source like so:
source: substringMatcher(self.get('data'))
Working solution here
Update
Updating my answer based on your updated question.
OK, so this is getting a little more complicated. You now need more than just one property (latitude) from the object. You need category_name and merchant name.
In addition to mapBy, which just grabs one property out of array, Ember also has map which lets you transform the array into pretty much anything you want to:
lats: function(){
var merchantsArr = this.get('model').map(function(thing){
var category_name = thing.category_name;
return thing.merchants.map(function(merchant){
return {
"name": merchant.name,
"category": category_name
};
});
});
merchantsArr = this.flatten(merchantsArr);
return merchantsArr;
}.property('model')
The code above looks complicated, but it's basically just returning an array of top level objects' merchants accompanied by category_name. Since this is an array of arrays, we will need to flatten it.
Then, inside the component, we need to keep in mind that we are not just passing in an array of strings, but rather we are passing in an array of objects. Therefore, we need to look through object's properties (name and category) for a match
$.each(strs, function(i, str) {
if (substrRegex.test(str.name) || substrRegex.test(str.category)) {
matches.push(str);
}
});
Lastly, to actually display both category and merchant name, you need to tell Typeahead how to do that:
templates: {
suggestion: Handlebars.compile('<p>{{name}} – {{category}}</p>')
}
Working solution here
I have a basic Json question - I have a JSON file. Every object in this file has columns repeated.
[
{
id: 1,
name: "ABCD"
},
{
id: 2,
name: "ABCDE"
},
{
id: 3,
name: "ABCDEF"
}
]
For optimization I was thinking to remove repeated column names.
{
"cols": [
"id",
"name"
],
"rows": [
[
"1",
"ABCD"
],
[
"2",
"ABCDE"
]
]
}
What I am trying to understand is - is this a better approach? Are there any disadvantages of this format? Say for writing unit tests?
EDIT
The second case (after your editing) is valid json. You can derive it to the following class using json2csharp
public class RootObject
{
public List<string> cols { get; set; }
public List<List<string>> rows { get; set; }
}
The very important point to note about a valid json is that it has no other way but to repeat the column names (or, keys in general) to represent values in json. You can test the validity of your json putting it # jsonlint.com
But if you want to optimize json by compressing it using some compression library like gzip (likewise), then I would recommend Json.HPack.
According to this format, it has many compression levels ranging from 0 to 4 (4 is the best).
At compression level 0:
you have to remove keys (property names) from the structure creating a header on index 0 with each property name. Then your compressed json would look like:
[
[
"id",
"name"
],
[
1,
"ABCD"
],
[
2,
"ABCDE"
],
[
3,
"ABCDEF"
]
]
In this way, you can compress your json at any levels as you want. But in order to work with any json library, you must have to decompress it to valid json first like the one you provided earlier with repeated property names.
For your kind information, you can have a look at the comparison between different compression techniques:
{
"cols": [
"id",
"name"
],
"rows": [
"1",
"ABCD"
], [
"2",
"ABCDE"
], [
"3",
"ABCDEF"
]
}
In this approach it will be hard to determine which value stand for which item (id,name). Your first approach was good if you use this JSON for communication.
A solution for it, is use any type (by your preference) of Object-Relational-Mapper,
By that, you can compress your JSON data and still using legible structure/code.
Please, see this article: What is "compressed JSON"?