sails.js query json objects inside table column - mysql

So this must be a strange question, I want sails.js ORM to search something like following
If this is the result for query for the following
Venue.findOne({id: 125274827508536}).exec()
returns >
{
"id": "125274827508536",
"attire": "Casual",
"can_post": false,
"category": "Restaurant/cafe",
"category_list": [
{
"id": "200742186618963",
"name": "Vegetarian & Vegan Restaurant"
},
{
"id": "192108214153222",
"name": "Breakfast & Brunch Restaurant"
},
{
"id": "188296324525457",
"name": "Sandwich Shop"
}
],
"checkins": 562,
"cover": {
"cover_id": 356427064393310,
"offset_x": 0,
"offset_y": 13,
"source": "https://fbcdn-sphotos-c-a.akamaihd.net/hphotos-ak-xaf1/t31.0-8/s720x720/460144_356427064393310_1179113344_o.jpg",
"id": "356427064393310"
},
"culinary_team": "Ramy Abu-Yousef : Owner, Chef\nDallas Jones: Chef",
"description": "Unique Sandwiches\nDelicious Salads\nHomemade Soups (4 daily)\nFresh Fruit Smoothies\nMilkshakes\nMOUSTACHE WALL OF FAME",
"general_manager": "Ramy Abu-Yousef & Syndey Friedemann",
"has_added_app": false,
"hours": {
"mon_1_open": "08:00",
"mon_1_close": "22:00",
"tue_1_open": "08:00",
"tue_1_close": "22:00",
"wed_1_open": "08:00",
"wed_1_close": "22:00",
"thu_1_open": "08:00",
"thu_1_close": "22:00",
"fri_1_open": "08:00",
"fri_1_close": "22:00",
"sat_1_open": "08:00",
"sat_1_close": "22:00",
"sun_1_open": "08:00",
"sun_1_close": "22:00"
},
"is_community_page": false,
"is_published": true,
"likes": 540,
"link": "https://www.facebook.com/JohnnyBarrs",
"location": {
"city": "Queenstown",
"country": "New Zealand",
"latitude": -45.032691433795,
"longitude": 168.66154298959,
"street": "15 Church Street",
"zip": "9300"
},
"name": "Johnny Barr's",
"parking": {
"lot": 1,
"street": 1,
"valet": 0
},
"payment_options": {
"amex": 1,
"cash_only": 0,
"discover": 0,
"mastercard": 1,
"visa": 1
},
"phone": "+64 (0)3 409 0169",
"price_range": "$$ (10-30)",
"restaurant_services": {
"delivery": 1,
"catering": 0,
"groups": 1,
"kids": 1,
"outdoor": 0,
"reserve": 0,
"takeout": 1,
"waiter": 0,
"walkins": 1
},
"restaurant_specialties": {
"breakfast": 1,
"coffee": 1,
"dinner": 1,
"drinks": 1,
"lunch": 1
},
"talking_about_count": 2,
"username": "JohnnyBarrs",
"website": "www.johnnybarrs.com",
"were_here_count": 562
}
Now what I need sails to do is the following
Venue.findOne({'restaurant_services': {'delivery': 1}).exec()
To return the same object as I have shown above,
Any thoughts please ?

If your adapter is mongo then you can do this out of the box
Venue.findOne({'restaurant_services.delivery': 1}).exec()
So you should consider your options if you can transform the data into another source to preform this query.
If not and your adapter is SQL based then it is more difficult and would require more information on your use case to decide on the most efficient option.
For instance if you could limit your query to a few indexed fields in a SQL database, you could then use lodash to find your records within your records.
Venue.find({/*limiting criteria to bring down the number of results*/}).exec(function(err,results){/* JSON.parse(results) then use lodash to find the final result*/})
Another option could be
Venue.find({restaurant_services: {contains: 'delivery: 1'}}).exec()
that is an out of the box idea, but one that might work.
Again depends deeply on your setup, how its indexed and out of how many venues that one would need to be found.

Related

Convert JSON list to table where one record is a list

I have a JSON list with 253 entries and 7 records where the 7th record is a list with usually 2 entries.
I'm trying to convert this in Power Query to give me a table output. I've only really ever used the basic connection in Excel to do this automatically without problems.
My current error is:
Expression.Error: We cannot convert a value of type Record to type Text.
Details:
Value=Record
Type=Type
I'm not sure where to go and I can't see examples for my specific situation, although I'm sure there are plenty of examples - just that I'm not good enough with this to understans
[
{
"Id": "lorum-ipsum1",
"Description": "sitename 1",
"Latitude": 1.0,
"Longitude": -1.0,
"Postcode": "AB1 2CD",
"CountryCode": "GB",
"Connectors": [
{
"Id": "lorum-ipsum1-a",
"Number": 1,
"Status": 1,
"ErrorCode": "NoError",
"ChargepointName": "GP00000",
"Shape": 0,
"Mode": 0,
"Volts": 240,
"Amps": 0,
"Phase": 2,
"PricingInformationUrl": "hrefhere",
"UsageRestrictions": []
},
{
"Id": "lorum-ipsum1-b",
"Number": 2,
"Status": 1,
"ErrorCode": "NoError",
"ChargepointName": "GP0000",
"Shape": 0,
"Mode": 0,
"Volts": 240,
"Amps": 0,
"Phase": 2,
"PricingInformationUrl": "hrefhere",
"UsageRestrictions": []
}
]
},
{
"Id": "lorum-ipsum2",
"Description": "sitename 2",
"Latitude": 1.0,
"Longitude": -1.0,
"Postcode": "AB1 2CD",
"CountryCode": "GB",
"Connectors": [
{
"Id": "lorum-ipsum2-a",
"Number": 1,
"Status": 1,
"ErrorCode": "NoError",
"ChargepointName": "GP00000",
"Shape": 0,
"Mode": 0,
"Volts": 240,
"Amps": 0,
"Phase": 2,
"PricingInformationUrl": "hrefhere",
"UsageRestrictions": []
},
{
"Id": "lorum-ipsum2-b",
"Number": 2,
"Status": 1,
"ErrorCode": "NoError",
"ChargepointName": "GP0000",
"Shape": 0,
"Mode": 0,
"Volts": 240,
"Amps": 0,
"Phase": 2,
"PricingInformationUrl": "hrefhere",
"UsageRestrictions": []
}
]
},
I think you want to use Table.FromRecords function to transform the JSON into a table. However, since the Connectors field of each record is itself a list of records, I think you will need to call it a second time.
To give you an example:
let
serialised = "[{""Id"":""lorum-ipsum1"",""Description"":""sitename 1"",""Latitude"":1,""Longitude"":-1,""Postcode"":""AB1 2CD"",""CountryCode"":""GB"",""Connectors"":[{""Id"":""lorum-ipsum1-a"",""Number"":1,""Status"":1,""ErrorCode"":""NoError"",""ChargepointName"":""GP00000"",""Shape"":0,""Mode"":0,""Volts"":240,""Amps"":0,""Phase"":2,""PricingInformationUrl"":""hrefhere"",""UsageRestrictions"":[]},{""Id"":""lorum-ipsum1-b"",""Number"":2,""Status"":1,""ErrorCode"":""NoError"",""ChargepointName"":""GP0000"",""Shape"":0,""Mode"":0,""Volts"":240,""Amps"":0,""Phase"":2,""PricingInformationUrl"":""hrefhere"",""UsageRestrictions"":[]}]},{""Id"":""lorum-ipsum2"",""Description"":""sitename 2"",""Latitude"":1,""Longitude"":-1,""Postcode"":""AB1 2CD"",""CountryCode"":""GB"",""Connectors"":[{""Id"":""lorum-ipsum2-a"",""Number"":1,""Status"":1,""ErrorCode"":""NoError"",""ChargepointName"":""GP00000"",""Shape"":0,""Mode"":0,""Volts"":240,""Amps"":0,""Phase"":2,""PricingInformationUrl"":""hrefhere"",""UsageRestrictions"":[]},{""Id"":""lorum-ipsum2-b"",""Number"":2,""Status"":1,""ErrorCode"":""NoError"",""ChargepointName"":""GP0000"",""Shape"":0,""Mode"":0,""Volts"":240,""Amps"":0,""Phase"":2,""PricingInformationUrl"":""hrefhere"",""UsageRestrictions"":[]}]}]",
deserialised = Json.Document(serialised),
toTable = Table.FromRecords(deserialised),
transformConnectors = Table.TransformColumns(toTable, {{"Connectors", Table.FromRecords}})
in
transformConnectors
which gives me:
From there, you can explore/continue the rest of the transformation yourself (since you haven't specified what you want).
You can expand some/all nested columns inside of the Connectors column by either clicking the icon highlighted in the image above -- or writing any necessary M code.

parse json output for primary and secondary hosts from replSetGetStatus

I've used pymongo to connect to mongo replica set and print the status of replica set using json dump. I want to parse this output and display "name" and "stateStr" into a list or array for the user to be able to pick a particular host.Here is my json dump output:
{
{
"replSetGetStatus": {
"date": "2016-10-07T14:21:25",
"members": [
{
"_id": 0,
"health": 1.0,
"name": "xxxxxxxxxxx:27017",
"optime": null,
"optimeDate": "2016-10-07T13:50:11",
"self": true,
"state": 1,
"stateStr": "PRIMARY",
"uptime": 32521
},
{
"_id": 1,
"health": 1.0,
"lastHeartbeat": "2016-10-07T14:21:24",
"lastHeartbeatRecv": "2016-10-07T14:21:24",
"name": "xxxxxxxxxxxx:27017",
"optime": null,
"optimeDate": "2016-10-07T13:50:11",
"pingMs": 0,
"state": 2,
"stateStr": "SECONDARY",
"syncingTo": "xxxxxxxxxxxx:27017",
"uptime": 27297
},
{
"_id": 2,
"health": 1.0,
"lastHeartbeat": "2016-10-07T14:21:24",
"lastHeartbeatRecv": "2016-10-07T14:21:24",
"name": "xxxxxxxxxxxxx:27020",
"pingMs": 0,
"state": 7,
"stateStr": "ARBITER",
"uptime": 32517
}
],
"myState": 1,
"ok": 1.0,
"set": "replica1"
}
}
Please try below Javascript code. It worked for me.
use admin;
var result = rs.status();
var length = result.members.length;
for (var i=0;i<length;i++){
print ("Server Name-" +result.members[i].name);
print ("Server State-" +result.members[i].stateStr);
}

N1QL query to filter JSON array in Couchbase server

I have the following array of data inside the data bucket SITES in my Couchbase server
"siteMaster": [
{
"sitename": "HTS_SITE_001",
"sitelink": "http://facebook.com",
"address" : "19/2, Bellandur, Bangalore India",
"filename": "site1.json",
"persons": 1,
"status": "70%",
"contact": "max.smith#honeywell.com",
}, {
"sitename": "HTS_SITE_002",
"sitelink": "http://facebook.com",
"address": "5th Avenue, New York",
"filename": "site2.json",
"persons": 1,
"status": "70%",
"contact": "john.smith#facebook.com",
}, {
"sitename": "HTS_SITE_003",
"sitelink": "http://facebook.com",
"address": "Palo Alto, California",
"filename": "site3.json",
"persons": 1,
"status": "80%",
"contact": "steve.jobs#apple.com",
}, {
"sitename": "HTS_SITE_004",
"sitelink": "http://facebook.com",
"address": "Bellandur, Bangalore",
"filename": "site4.json",
"persons": 1,
"status": "80%",
"contact": "max.mustermann#deutsche.com",
}
]
The N1QL query for
select * from SITES where status = "70%" should return me two rows, but unfortunately it is not returning any rows.
Where am I going wrong with the query ?
Please use the following query:
SELECT *
FROM SITES
WHERE ANY sm IN siteMaster SATISFIES sm.status = "70%" END;
You can also create the following array index to speed up the query:
CREATE INDEX idx ON SITES( DISTINCT ARRAY sm.status FOR sm IN siteMaster END );

trouble with parseing JSON with ASPJSON for classic ASP

I am using ASP JSON at http://www.aspjson.com/
I am trying to parse the salehistory the out of this JSON response below. I can get all the nodes under the property object like this
oJSON.data("property").item(0).item("vintage").item("lastModified")
However when I try to go deeper I get errors
I have tried
oJSON.data("property").item(0).item("salehistory").item("salesearchdate") but that does not return anything.
I think that it has something to do with the fact that property is an object and SalesHistory is an object inside of Property however I cannot seem to get the values out of the salehistory level.
Below is the JSON structure that I am trying to parse. Any help would be appreciated. .
{
"status": {
"version": "1.0.0",
"code": 0,
"msg": "SuccessWithResult",
"total": 1,
"page": 1,
"pagesize": 10
},
"property": [
{
"identifier": {
"obPropId": 3464768712115,
"fips": "12115",
"apn": "0283080001",
"apnOrig": "0283080001"
},
"lot": {
"lotSize1": 0.837
},
"address": {
"country": "US",
"countrySubd": "FL",
"line1": "7580 PRESERVATION DR",
"line2": "SARASOTA, FL 34241",
"locality": "Sarasota",
"matchCode": "ExaStr",
"oneLine": "7580 PRESERVATION DR, SARASOTA, FL 34241",
"postal1": "34241",
"postal2": "5201",
"postal3": "R038"
},
"location": {
"accuracy": "Street",
"elevation": 0,
"latitude": "27.267342",
"longitude": "-82.419812",
"distance": 0,
"geoid": "MT30003379,RS0000548079,SD67554,SS156496,SS156498,SS190868"
},
"summary": {
"propclass": "Single Family Residence / Townhouse",
"propsubtype": "SINGLE FAMILY",
"proptype": "SFR",
"yearbuilt": 2005,
"propLandUse": "SFR"
},
"building": {
"size": {
"universalsize": 4256
},
"rooms": {
"bathstotal": 5,
"beds": 4
}
},
"vintage": {
"lastModified": "2015-9-11",
"pubDate": "2015-10-7"
},
"salehistory": [
{
"salesearchdate": "2009-3-30",
"saleTransDate": "2009-3-30",
"amount": {
"saleamt": 1250000,
"salerecdate": "2009-3-30",
"saledisclosuretype": 0,
"saledocnum": "37737",
"saletranstype": "Resale"
},
"calculation": {
"priceperbed": 312500,
"pricepersizeunit": 294
}
},
{
"salesearchdate": "2005-8-9",
"saleTransDate": "2005-8-9",
"amount": {
"saleamt": 185000,
"salerecdate": "2005-8-15",
"saledisclosuretype": 0,
"saledocnum": "181999",
"saletranstype": "Resale"
},
"calculation": {
"priceperbed": 46250,
"pricepersizeunit": 43
}
}
]
}
]
}
Actually I was able to get it figured out.
Since it was a nested object within the property object I need to do the following to access it.
response.write oJSON.data("property").item(0).item("salehistory").item(0).item("amount").item("saleamt"
Adding .item(0) after sales history allowed me to access the salehistory object

How to convert a json file without the same length of json objects into csv

I have a json file and I want to convert it to csv format.
The problem I face is that every json object in the file has not the same length of the converted columns I have. For example the one object have 49 columnns and the next have 50.
I provide here an example of 2 data from which the first one has not the creator.slug but the next has it is and so there is the problem with data. The problem is that the process create all 50 columns but for the object which don't have the value creator.slug it takes the next price.
{
"id": 301852363,
"name": "Song of the Sea",
"blurb": "One evening, two shows: SIRENS and The Girl From Bare Cove. Building a community. Giving voice to survivors of sexual violence.",
"goal": 5000,
"pledged": 671,
"state": "live",
"slug": "song-of-the-sea",
"disable_communication": false,
"country": "US",
"currency": "USD",
"currency_symbol": "$",
"currency_trailing_code": true,
"deadline": 1399293386,
"state_changed_at": 1397133386,
"created_at": 1396672480,
"launched_at": 1397133386,
"backers_count": 20,
"photo": {
"full": "https://s3.amazonaws.com/ksr/projects/939387/photo-full.jpg?1397874930",
"ed": "https://s3.amazonaws.com/ksr/projects/939387/photo-ed.jpg?1397874930",
"med": "https://s3.amazonaws.com/ksr/projects/939387/photo-med.jpg?1397874930",
"little": "https://s3.amazonaws.com/ksr/projects/939387/photo-little.jpg?1397874930",
"small": "https://s3.amazonaws.com/ksr/projects/939387/photo-small.jpg?1397874930",
"thumb": "https://s3.amazonaws.com/ksr/projects/939387/photo-thumb.jpg?1397874930",
"1024x768": "https://s3.amazonaws.com/ksr/projects/939387/photo-1024x768.jpg?1397874930",
"1536x1152": "https://s3.amazonaws.com/ksr/projects/939387/photo-1536x1152.jpg?1397874930"
},
"creator": {
"id": 1714048992,
"name": "Maridee Slater",
"slug": "maridee",
"avatar": {
"thumb": "https://s3.amazonaws.com/ksr/avatars/996153/DSC_0310.thumb.jpg?1337713264",
"small": "https://s3.amazonaws.com/ksr/avatars/996153/DSC_0310.small.jpg?1337713264",
"medium": "https://s3.amazonaws.com/ksr/avatars/996153/DSC_0310.medium.jpg?1337713264"
},
"urls": {
"web": {
"user": "https://www.kickstarter.com/profile/maridee"
},
"api": {
"user": "https://api.kickstarter.com/v1/users/1714048992?signature=1398256877.e6d63adcca055cd041a5920368b197d40459f748"
}
}
},
"location": {
"id": 2459115,
"name": "New York",
"slug": "new-york-ny",
"short_name": "New York, NY",
"displayable_name": "New York, NY",
"country": "US",
"state": "NY",
"urls": {
"web": {
"discover": "https://www.kickstarter.com/discover/places/new-york-ny",
"location": "https://www.kickstarter.com/locations/new-york-ny"
},
"api": {
"nearby_projects": "https://api.kickstarter.com/v1/discover?signature=1398256786.89b2c4539aeab4ad25982694dd7e659e8c12028f&woe_id=2459115"
}
}
},
"category": {
"id": 17,
"name": "Theater",
"slug": "theater",
"position": 14,
"urls": {
"web": {
"discover": "http://www.kickstarter.com/discover/categories/theater"
}
}
},
"urls": {
"web": {
"project": "https://www.kickstarter.com/projects/maridee/song-of-the-sea"
}
}
},
{
"id": 967108708,
"name": "Good Bread Alley",
"blurb": "A play by April Yvette Thompson. A Gullah Healer Woman and an Afro-Cuban Priest forge a new world of magic & dreams in Jim Crow Miami.",
"goal": 100000,
"pledged": 33242,
"state": "live",
"slug": "good-bread-alley",
"disable_communication": false,
"country": "US",
"currency": "USD",
"currency_symbol": "$",
"currency_trailing_code": true,
"deadline": 1399271911,
"state_changed_at": 1396334313,
"created_at": 1393278556,
"launched_at": 1396334311,
"backers_count": 261,
"photo": {
"full": "https://s3.amazonaws.com/ksr/projects/883489/photo-full.jpg?1397869394",
"ed": "https://s3.amazonaws.com/ksr/projects/883489/photo-ed.jpg?1397869394",
"med": "https://s3.amazonaws.com/ksr/projects/883489/photo-med.jpg?1397869394",
"little": "https://s3.amazonaws.com/ksr/projects/883489/photo-little.jpg?1397869394",
"small": "https://s3.amazonaws.com/ksr/projects/883489/photo-small.jpg?1397869394",
"thumb": "https://s3.amazonaws.com/ksr/projects/883489/photo-thumb.jpg?1397869394",
"1024x768": "https://s3.amazonaws.com/ksr/projects/883489/photo-1024x768.jpg?1397869394",
"1536x1152": "https://s3.amazonaws.com/ksr/projects/883489/photo-1536x1152.jpg?1397869394"
},
"creator": {
"id": 749318998,
"name": "April Yvette Thompson",
"avatar": {
"thumb": "https://s3.amazonaws.com/ksr/avatars/9751919/kick_thumb.thumb.jpg?1396128151",
"small": "https://s3.amazonaws.com/ksr/avatars/9751919/kick_thumb.small.jpg?1396128151",
"medium": "https://s3.amazonaws.com/ksr/avatars/9751919/kick_thumb.medium.jpg?1396128151"
},
"urls": {
"web": {
"user": "https://www.kickstarter.com/profile/749318998"
},
"api": {
"user": "https://api.kickstarter.com/v1/users/749318998?signature=1398256877.af4db50c53f93339b05c7813f4534e833eaca270"
}
}
},
"location": {
"id": 2459115,
"name": "New York",
"slug": "new-york-ny",
"short_name": "New York, NY",
"displayable_name": "New York, NY",
"country": "US",
"state": "NY",
"urls": {
"web": {
"discover": "https://www.kickstarter.com/discover/places/new-york-ny",
"location": "https://www.kickstarter.com/locations/new-york-ny"
},
"api": {
"nearby_projects": "https://api.kickstarter.com/v1/discover?signature=1398256786.89b2c4539aeab4ad25982694dd7e659e8c12028f&woe_id=2459115"
}
}
},
"category": {
"id": 17,
"name": "Theater",
"slug": "theater",
"position": 14,
"urls": {
"web": {
"discover": "http://www.kickstarter.com/discover/categories/theater"
}
}
},
"urls": {
"web": {
"project": "https://www.kickstarter.com/projects/749318998/good-bread-alley"
}
}
}
Here is the code I run
#open the json file
require(RJSONIO)
require(rjson)
library("rjson")
filename2 <- "C:/Users/Desktop/in.json"
json_data <- fromJSON(file = filename2)
#unlist the json because it has a problem
unlisted <- unlist(unlist(json_data,recursive=FALSE),recursive=FALSE)
use to fill the NA but as I can understand now it is for already existed nulls http://stackoverflow.com/questions/16947643/getting-imported-json-data-into-a-data-frame-in-r/16948174#16948174
unlisted <- lapply(unlisted, function(x) {
x[sapply(x, is.null)] <- NA
unlist(x)
})
json <- do.call("rbind", unlisted)
Here is a full list with the columns of the output csv and after that I provide what I would like to keep from every object of json, less columns
id
name
blurb
goal
pledged
state
slug
disable_communication
country
currency
currency_symbol
currency_trailing_code
deadline
state_changed_at
created_at
launched_at
backers_count
photo.full
photo.ed
photo.med
photo.little
photo.small
photo.thumb
photo.1024x768
photo.1536x1152
creator.id
creator.name
creator.slug
creator.avatar.thumb
creator.avatar.small
creator.avatar.medium
creator.urls.web.user
creator.urls.api.user
location.id
location.name
location.slug
location.short_name
location.displayable_name
location.country
location.state
location.urls.web.discover
location.urls.web.location
location.urls.api.nearby_projects
category.id
category.name
category.slug
category.position
category.urls.web.discover
category.urls.web.project
category.urls.web.rewards
Here it is the list of columns I would try to have in the output csv:
id
name
blurb
goal
pledged
state
slug
disable_communication
country
currency
currency_symbol
currency_trailing_code
deadline
state_changed_at
created_at
launched_at
backers_count
creator.id
creator.name
creator.slug
location.id
location.name
location.slug
location.short_name
location.displayable_name
location.country
location.state
category.id
category.name
category.slug
category.position
Looks like there's a very similar question (with answer, though not pure R) here: convert json to csv format
However, since you do seem to want most, if not all, the JSON in a "wide CSV" format you can use fromJSON from jsonlite, rbindlist from data.table (which gets you the fill=TRUE parameter to handle uneven lists nicely) and unlist:
library(jsonlite)
library(data.table)
# tell fromJSON we want a list back
json_data <- fromJSON("in.json", simplifyDataFrame=FALSE)
# iterate over the list we have so we can "flatten" it then
# covert it back to a data.frame-like object
dat <- rbindlist(lapply(json_data, function(x) {
as.list(unlist(x))
}), fill=TRUE)
You may need to tweak column names, but I think this gets you what you're looking for.