Serialize the response from backend to store ember store - json

My response from backend is not in form which ember store. I am not able to serialize the response.
response.json
[{
"pk": 127,
"url": "http://example.com/api/galleries/127/",
"gallery_name": "Faces",
"thumbnail_url": "https://example.cloud.net/galleryThumbs/2656a05c-4ec7-3eea-8c5e-d8019454d443.jpg",
"time": "1 month ago",
"description": "Created by user",
"is_following": true,
"feedPhotos": [{
"pk": 624,
"url": "http://example.com/api/photos/624/",
"profilePic": "https://example.cloud.net/userDPs/50906ce2-394d-39c8-9261-8cf78e3611c2.jpg",
"userName": "Nabeela",
"userKarma": 915,
"caption": "Old woman spinning her 'chhos-khor' ...a rotation of which is equivalent to the recitation of a mantra.",
"numComments": 0,
"owner": "http://example.com/api/users/44/",
"time": "1 month ago",
"photo_url": "https://example.cloud.net/photos/9cbd6423-3bc5-36e0-b8b4-d725efb3249a.jpg",
"comments_url": "http://example.com/api/photos/624/comments/",
"numFives": 4,
"fivers_url": "http://example.com/api/photogalleries/1362/fivers/",
"fivers_pk": 1362,
"fullphoto_url": "http://example.com/api/photogalleries/1362/photo/",
"fullphoto_pk": 1362,
"is_fived": true,
"hiFiveKarma": 1,
"owner_pk": 44,
"userFirstName": "Nabeela",
"is_bookmarked": false
}, {
"pk": 574,
"url": "http://example.com/api/photos/574/",
"profilePic": "https://example.cloud.net/userDPs/b6f69e4e-980d-3cc3-8b3e-3eb1a7f21350.jpg",
"userName": "Rohini",
"userKarma": 194,
"caption": "Life # Myanmar!",
"numComments": 0,
"owner": "http://example.com/api/users/45/",
"time": "2 months ago",
"photo_url": "https://example.cloud.net/photos/eeae72d5-d6af-391e-a218-b442c0c7e34e.jpg",
"comments_url": "http://example.com/api/photos/574/comments/",
"numFives": 2,
"fivers_url": "http://example.com/api/photogalleries/1303/fivers/",
"fivers_pk": 1303,
"fullphoto_url": "http://example.com/api/photogalleries/1303/photo/",
"fullphoto_pk": 1303,
"is_fived": false,
"hiFiveKarma": 0,
"owner_pk": 45,
"userFirstName": "Rohini",
"is_bookmarked": false
}
]
}, {
"pk": 65,
"url": "http://example.com/api/galleries/65/",
"gallery_name": "Royal",
"thumbnail_url": "https://example.cloud.net/galleryThumbs/d8a900af-1f1d-3977-8cc8-b8bb36e32be5.jpg",
"time": "2 months ago",
"description": "This is a gallery about Royal",
"is_following": false,
"feedPhotos": [{
"pk": 347,
"url": "http://example.com/api/photos/347/",
"profilePic": "https://example.cloud.net/userDPs/50906ce2-394d-39c8-9261-8cf78e3611c2.jpg",
"userName": "Nabeela",
"userKarma": 915,
"caption": "I cannot forget the name of this palace - Moti Mahal (translation: Pearl Palace). Indescribably beautiful, ainnit! at Mehrangarh fort, Jodhp",
"numComments": 0,
"owner": "http://example.com/api/users/44/",
"time": "2 months ago",
"photo_url": "https://example.cloud.net/photos/958ed406-708e-3f01-a2f4-9467cd709fdd.jpg",
"comments_url": "http://example.com/api/photos/347/comments/",
"numFives": 4,
"fivers_url": "http://example.com/api/photogalleries/759/fivers/",
"fivers_pk": 759,
"fullphoto_url": "http://example.com/api/photogalleries/759/photo/",
"fullphoto_pk": 759,
"is_fived": false,
"hiFiveKarma": 0,
"owner_pk": 44,
"userFirstName": "Nabeela",
"is_bookmarked": false
}, {
"pk": 593,
"url": "http://example.com/api/photos/593/",
"profilePic": "https://example.cloud.net/userDPs/95ac6974-f7df-338c-ab84-99fa1df7514c.jpg",
"userName": "Vikanshu",
"userKarma": 932,
"caption": "Marvelous architecture!! in Florence, Italy",
"numComments": 0,
"owner": "http://example.com/api/users/48/",
"time": "1 month ago",
"photo_url": "https://example.cloud.net/photos/7a86eb37-6c68-3d6c-b6cf-2e3b74d330dd.jpg",
"comments_url": "http://example.com/api/photos/593/comments/",
"numFives": 4,
"fivers_url": "http://example.com/api/photogalleries/1363/fivers/",
"fivers_pk": 1363,
"fullphoto_url": "http://example.com/api/photogalleries/1363/photo/",
"fullphoto_pk": 1363,
"is_fived": false,
"hiFiveKarma": 0,
"owner_pk": 48,
"userFirstName": "Vikanshu",
"is_bookmarked": false
}]
}]
How do I serialize this using JSONPISerailizer or any other serializer in ember-cli so that it gets stored in ember store

Reference jsonapi.org
++++Top Level:
Root:
A JSON object must be root of every JSON API request response.
A document must contain at least one top-level members:
1. data: documents "primary data"
2. errors: an array of error objects (id,status,code,title....)
3. meta: a meta object that contains non-standard meta-information (copyright,author...)
member data and errors must not co-exist together.
"data"{}
+++++Resource Objects
1. A resource object MUST contain atleast following top-level member
*id
*type
```
//structure-1
//for galleries
{
"data": {
"type": "galleries",
"id": "1"
}
}
//for photos
{
"data": {
"type": "photos",
"id": "1"
}
}
```
In addition, a resource object may contain any of these top-level members
*attributes
*relationship
*links
*meta
//added attributes first
```
//structure-2
//for galleries
{
"data": {
"type": "galleries",
"id": "1",
"attributes": {
"galleryName": "Faces"
"thumbnailUrl:"https://example.cloud.net/galleryThumbs/2656a05c-4ec7.jpg",
"description": "Created by user",
}
}
}
//for photos
{
"data": {
"type": "photos",
"id": "1",
"attributes":{
userName: "Nabeela",
userKarma: 915
}
}
}
```
//Adding relationship
Relationship object must contain atleast one of the following
*links (containing atleast one of "self" or "related" resource link
*data
*meta
//link in relationship (minimum one required from link,data,meta).
//
```
//structure-3
//for galleries
{
"data":[{ //Array(square bracket as adding relationship one more item to data
"type": "galleries",
"id": "1",
"attributes": {
"galleryName": "Faces"
"thumbnailUrl:"https://example.cloud.net/galleryThumbs/2656a05c-4ec7.jpg",
"description": "Created by user",
},
"relationships":{
"links":{
"self": "https://example.cloud.net/photos/9cbd6423.jpg //"photo_url" in your payload
},
}]
}
}
```
//data in relationship
```
//structure-4
//for galleries
{
"data": [{
"type": "galleries",
"id": "1",
"attributes": {
"galleryName": "Faces"
"thumbnailUrl:"https://example.cloud.net/galleryThumbs/2656a05c-4ec7.jpg",
"description": "Created by user",
},
"relationships":{ //This has all your photo stuff
"links":{
"self": "https://example.cloud.net/photos/9cbd6423.jpg //"photo_url" in your payload
},
"data": { //picked it up from structure-1
"type": "photos",
"id": "77"
}
}]
}
}
```
//Adding related resource "included"
```
//for galleries
{
"data": [{
"type": "galleries",
"id": "1",
"attributes": {
"galleryName": "Faces"
"thumbnailUrl:"https://example.cloud.net/galleryThumbs/2656a05c-4ec7.jpg",
"description": "Created by user",
},
"relationships":{ //This has all your photo stuff
"links":{
"self": "https://example.cloud.net/photos/9cbd6423.jpg //"photo_url" in your payload
},
"data": { //picked it up from structure-1
"type": "photos",
"id": "77"
}
}],
"included":[{
"type": "photos",
"id": "77",
"attributes":{
userName: "Nabeela",
userKarma: 915
},
{
"type": "photos",
"id": "78",
"attributes":{
userName: "Nabeela",
userKarma: 915
}
}]
}
}
```
For collections. I am not confident but try this
Now for collection of galleries.
//for galleries
{
"data": [{
"type": "galleries",
"id": "1",
"attributes": {
"galleryName": "Faces"
"thumbnailUrl:"https://example.cloud.net/galleryThumbs/2656a05c-4ec7.jpg",
"description": "Created by user",
},
"relationships":{ //This has all your photo stuff
"links":{
"self": "https://example.cloud.net/photos/9cbd6423.jpg //"photo_url" in your payload
},
"data": { //picked it up from structure-1
"type": "photos",
"id": "77"
}
},{
"type": "galleries",
"id": "2",
"attributes": {
"galleryName": "Faces"
"thumbnailUrl:"https://example.cloud.net/galleryThumbs/2656a05c-4ec7.jpg",
"description": "Created by user",
},
"relationships":{ //This has all your photo stuff
"links":{
"self": "https://example.cloud.net/photos/9cbd6423.jpg //"photo_url" in your payload
},
"data": { //picked it up from structure-1
"type": "photos",
"id": "79"
}
}],
"included":[{
"type": "photos",
"id": "77",
"attributes":{
userName: "Nabeela",
userKarma: 915
},{
"type": "photos",
"id": "78",
"attributes":{
userName: "Nabeela",
userKarma: 915
},{
"type": "photos",
"id": "79",
"attributes":{
userName: "Nabeela",
userKarma: 915
}
}]
}
}
============Implementation part =================================
JSONSerializer normalization process follows these steps
*normalizeResponse : entry method.
*normalizeCreateRecordResponse : a normalizeResponse for specific operation.
*normalizeSingleResponse|normalizeArrayResponse:
- for methods like createRecord. we expect a single record back.
- for methods like findAll we expect multiple records back.
+normalize =
normalizeArray iterates and calls normalize for each of it's records
normalizeSingle call its once.
+extractID | extractAttributes | extractRelationships
= normalize delegates to these method to turn record payload into jsonAPI format
Starting with normalizeResponse method. If you open and see normalizeResponse method
in json-serializer
link normalizeResponse: https://github.com/emberjs/data/blob/v2.2.1/packages/ember-
data/lib/serializers/json-serializer.js#L192
you with find a switch case switch(requestType). If requestType if
"findRecord" then "normalizeFindRecordResponse" is called
"queryRecord" then "normalizeQueryRecordResponse" is called
"findAll" then "normalizeFindAllResponse" is called
...so on and so forth.
if you notice the parameter passed to all the methods are same as that of normalize
(...arguments) :)
**Lets start for findAll
i.e normalizeResponse -> normalizeFindAllResponse -> normalizeArrayResponse
as normalizeFindAllResponse method has only one line that call
normalizeArrayResponse.
normalizeFindAllResponse
normalizeResponse -> normalizeFindAllResponse -> normalizeArrayResponse ->
_normalizeResponse{ extractMeta,normalize }
extractMeta [extract meta information like pagination and stuff ]
if single: normalize []
example of normalize method in emberjs docs
```
import DS from 'ember-data';
export default DS.JSONSerializer.extend({
normalize: function(typeClass, hash) {
var fields = Ember.get(typeClass, 'fields');
fields.forEach(function(field) {
var payloadField = Ember.String.underscore(field);
if (field === payloadField) { return; }
hash[field] = hash[payloadField];
delete hash[payloadField];
});
return this._super.apply(this, arguments);
}
});
```
"normalizeArrayResponse calls `return this._normalizeResponse
(store,primaryModelClass,payload,id,requestType,false).
so isSingle is false for _normalizeResponse method. so we will have to push all the
related records of included array
in our case the photos which is done by below snippet from "_normalizeRespose"
method.
_normalizeResponse
```
else{
documentHash.data = payload.map((item) => {
let { data, included } = this.normalize(primaryModelClass,item);
if(included){
documentHash.included.push(...included);
}
return data;
});
return documentHash;
}
```
Things are still unclear in the context of our JSON reponse from server
but atleast we know the flow now.
Lets try to apply it for findAll ( as per the flow above).
run "ember g serializer application" //assuming you are using ember-cli and you
intend to make this serializer generic for application.
As of now I have no information how and when normalizeResponse is called. :(
I just scanned through and guess on recieving data from server the store calls
normalizeResponseHelpers which in turn calls normalizeResponse.
In any case "normalizeResponse" is going to send payload and other necessar
information to normalizeFindAllResponse(...arguments) which in turn will call
normalizeArrayResponse(...arguments) which in turn will call "_normalizeRespone".
Here is where we need to take action
for extraMeta and normalize.
+extraMeta
I am not sure if there is any meta information in you json response.
in case there is you can refer to the example mentioned in docs
extractMeta
So I guess you can directly user the normalize method from example in your application ;).
please try and check. Since i am learning ember myself I cannot guarantee it will work but it should. the lonngggg explation is my thought while i was learning the problem/solution
//app/serializers/application.js
+normalize
```
import DS from 'ember-data';
export default DS.JSONSerializer.extend({
normalize: function(typeClass, hash) {
var fields = Ember.get(typeClass, 'fields');
fields.forEach(function(field) {
var payloadField = Ember.String.underscore(field);
if (field === payloadField) { return; }
hash[field] = hash[payloadField];
delete hash[payloadField];
});
return this._super.apply(this, arguments);
}
});
```
The primary key in the JSON from server is pk. You will have to mention that too
http://emberjs.com/api/data/classes/DS.JSONSerializer.html#property_primaryKey
app/serializers/application.js
import DS from 'ember-data';
export default DS.JSONSerializer.extend({
primaryKey: 'pk'
});

Related

Swift, JSON model

I have a question regarding building a JSON model namely, I should declare a date that will be different each day - in this case: "2020-11-19" as below.
This is a response for the current day.
{
"dates": {
"2020-11-19": {
"countries": {
"Poland": {
"date": "2020-11-19",
"id": "poland",
"links": [
{
"href": "/api/2020-11-19/country/poland",
"rel": "self",
"type": "GET"
}
],
"name": "Poland",
"name_es": "Polonia",
"name_it": "Polonia",
"regions": [],
"source": "John Hopkins University",
"today_confirmed": 796798,
"today_deaths": 12088,
"today_new_confirmed": 23975,
"today_new_deaths": 637,
"today_new_open_cases": 4335,
"today_new_recovered": 19003,
"today_open_cases": 422824,
"today_recovered": 361886,
"today_vs_yesterday_confirmed": 0.03102262743215456,
"today_vs_yesterday_deaths": 0.05562832940354556,
"today_vs_yesterday_open_cases": 0.010358695210626712,
"today_vs_yesterday_recovered": 0.055421236981710864,
"yesterday_confirmed": 772823,
"yesterday_deaths": 11451,
"yesterday_open_cases": 418489,
"yesterday_recovered": 342883
}
},
"info": {
"date": "2020-11-19 00:00CET",
"date_generation": "2020-11-19 22:34",
"yesterday": "2020-11-18 00:00CET"
}
}
},
"metadata": {
"by": "Narrativa & AppliedXL",
"url": [
"wwww.narrativa.com",
"www.appliedxl.com"
]
},
"total": {
"date": "2020-11-19",
"name": "Total",
"name_es": "Total",
"name_it": "Total",
"rid": "#total",
"source": "Narrativa",
"today_confirmed": 56684618,
"today_deaths": 1356365,
"today_new_confirmed": 525111,
"today_new_deaths": 8186,
"today_new_open_cases": 273944,
"today_new_recovered": 242981,
"today_open_cases": 19082735,
"today_recovered": 36245518,
"today_vs_yesterday_confirmed": 0.009350349175964112,
"today_vs_yesterday_deaths": 0.00607189401407382,
"today_vs_yesterday_open_cases": 0.014564678824917632,
"today_vs_yesterday_recovered": 0.00674899660543371,
"yesterday_confirmed": 56159507,
"yesterday_deaths": 1348179,
"yesterday_open_cases": 18808791,
"yesterday_recovered": 36002537
},
"updated_at": "2020-11-19 21:34UTC"
}
How I should build a model so that this property would be changed every day? (of course, this date needs to have a proper format "yyyy-MM-dd")
I would probably do:
var dates: [String: YourModel]
if it will potentially have multiple dates in that field.. otherwise, I'd probably do a custom object that overrides init(from decoder: and parses out that info

Extracting data from nested JSON with python 3

Looking for a way to extract data from nested json.
The data was extracted from windows scheduler as xml and converted to json.
It represents OS scheduled tasks, there are about 100 tasks and my goal is to extract the information from json to an external DB.
This is what my json file looks like :
{
"Tasks": {
"Task": [
{
"RegistrationInfo": {
"Author": "Adobe Systems Incorporated",
"Description": "This task keeps your Adobe Reader and Acrobat applications up to date with the latest enhancements and security fixes",
"URI": "\\Adobe Acrobat Update Task"
},
"Principals": {
"Principal": {
"GroupId": "S-1-5-4",
"_id": "Author"
}
},
"Settings": {
"DisallowStartIfOnBatteries": "true",
"StopIfGoingOnBatteries": "true",
"MultipleInstancesPolicy": "IgnoreNew",
"StartWhenAvailable": "true",
"IdleSettings": {
"Duration": "PT10M",
"WaitTimeout": "PT1H",
"StopOnIdleEnd": "true",
"RestartOnIdle": "false"
}
},
"Triggers": {
"LogonTrigger": {
"StartBoundary": "2013-08-01T12:05:00",
"EndBoundary": "2027-05-02T08:00:00",
"Delay": "PT12M",
"Repetition": {
"Interval": "PT3H30M"
},
"_id": "TriggerUserLoggon"
},
"CalendarTrigger": {
"StartBoundary": "2013-01-01T09:00:00",
"EndBoundary": "2027-05-02T12:05:00",
"ScheduleByDay": {
"DaysInterval": "1"
},
"_id": "TriggerDaily"
}
},
"Actions": {
"Exec": {
"Command": "C:\\Program Files (x86)\\Common Files\\Adobe\\ARM\\1.0\\AdobeARM.exe"
},
"_Context": "Author"
},
"_xmlns": "http://schemas.microsoft.com/windows/2004/02/mit/task",
"_version": "1.2"
},
{
"RegistrationInfo": {
"Author": "Adobe",
"Description": "This task keeps your Adobe Flash NPAPI Player installation up to date with the latest enhancements and security fixes. If this task is disabled or removed, Adobe Flash Player will be unable to automatically secure your machine with the latest security fixes.",
"URI": "\\Adobe Flash Player NPAPI Notifier"
},
"Principals": {
"Principal": {
"UserId": "S-1-5-21-2755204513-1269241785-1912306605-1001",
"LogonType": "InteractiveToken",
"_id": "Author"
}
},
"Settings": {
"DisallowStartIfOnBatteries": "false",
"StopIfGoingOnBatteries": "true",
"MultipleInstancesPolicy": "IgnoreNew",
"StartWhenAvailable": "true",
"RunOnlyIfNetworkAvailable": "true",
"IdleSettings": {
"Duration": "PT10M",
"WaitTimeout": "PT1H",
"StopOnIdleEnd": "true",
"RestartOnIdle": "false"
}
},
"Triggers": {
"CalendarTrigger": {
"StartBoundary": "1999-12-31T16:14:00-08:00",
"Repetition": {
"Interval": "PT1H",
"Duration": "P1D"
},
"ScheduleByDay": {
"DaysInterval": "7"
},
"_id": "NotificationTrigger"
}
},
"Actions": {
"Exec": {
"Command": "C:\\Windows\\SysWOW64\\Macromed\\Flash\\FlashUtil32_32_0_0_330_Plugin.exe",
"Arguments": "-check plugin"
},
"_Context": "Author"
},
"_xmlns": "http://schemas.microsoft.com/windows/2004/02/mit/task",
"_version": "1.2"
}
The code below works for just one task, but i need to get the data from every task , i guess i need to find a way to insert the number of tasks to the value of index and loop them one by one ,but nothing i try works.
import json
index=1
with open('name.json', 'r') as f:
array = json.load(f)
ts=(array['Tasks'])
print('Author is',ts['Task'][index]['RegistrationInfo']['Author'])
print('Description is' ,ts['Task'][index]['RegistrationInfo']['Description'])
print('URI is',ts['Task'][index]['RegistrationInfo']['URI'])
print('user ID is', ts['Task'][index]['Principals']['Principal']['UserId'])
print('Logon Type is', ts['Task'][index]['Principals']['Principal']['LogonType'])
print(' ID is', ts['Task'][index]['Principals']['Principal']['_id'])
print(' DisallowStartIfOnBatteries is ', ts['Task'][index]['Settings']['DisallowStartIfOnBatteries'])
print(' StopIfGoingOnBatteries is ', ts['Task'][index]['Settings']['StopIfGoingOnBatteries'])
print(' MultipleInstancesPolicy is ', ts['Task'][index]['Settings']['MultipleInstancesPolicy'])
print(' StartWhenAvailable is ', ts['Task'][index]['Settings']['StartWhenAvailable'])
print(' RunOnlyIfNetworkAvailable is ', ts['Task'][index]['Settings']['RunOnlyIfNetworkAvailable'])

What is the best method to seeding a Node / MongoDB application?

So, I'm new to the MEAN stack, and I've hit a wall trying to seed MongoDB. I'm using Mongoose to communicate with the database, and there's a bunch of documentation suggesting I should be able to seed using populated JSON files.
What I've tried:
node-mongo-seed; Pretty straight forward, but consistently throws errors on the end of arrays. (Perhaps the missing bson module is at fault?)
{ [Error: Cannot find module '../build/Release/bson'] code: 'MODULE_NOT_FOUND' }
js-bson: Failed to load c++ bson extension, using pure JS version
Seeding files from directory /Users/Antwisted/code/wdi/MEAN/seeds
----------------------
Seeding collection locations
err = [SyntaxError: /Users/Antwisted/code/wdi/MEAN/seeds/locations.json: Unexpected token {]
mongoose-seed; Also pretty straight forward, basically puts the JSON objects into a variable before exporting to the database. Promising, but... more errors...
Successfully initialized mongoose-seed
[ 'app/models/locationsModel.js' ]
Locations collection cleared
Error creating document [0] of Location model
Error: Location validation failed
Error creating document [1] of Location model
Error: Location validation failed
Error creating document [2] of Location model
Error: Location validation failed...
So, my thoughts were that it was probably a syntax error within the JSON structure, but playing around with that has not yielded any real solutions (or maybe I'm missing it?). Sample of my JSON:
{
{
"header": "Dan's Place",
"rating": 3,
"address": "125 High Street, New York, 10001",
"cord1": -73.0812,
"cord2": 40.8732,
"attributes": ["Hot drinks", "Food", "Premium wifi"],
"hours": [
{
"days": "Monday - Friday",
"hours": "7:00am - 7:00pm",
"closed": false
},
{
"days": "Saturday",
"hours": "8:00am - 5:00pm",
"closed": false
},
{
"days": "Sunday",
"closed": true
}
],
"reviews": [
{
"rating": 4,
"id": ObjectId(),
"author": "Philly B.",
"timestamp": "new Date('Feb 3, 2016')",
"body": "It was fine, but coffee was a bit dull. Nice atmosphere."
},
{
"rating": 3,
"id": ObjectId(),
"author": "Tom B.",
"timestamp": "new Date('Feb 23, 2016')",
"body": "I asked for her number. She said no."
}
]
},
{
"header": "Jared's Jive",
"rating": 5,
"address": "747 Fly Court, New York, 10001",
"cord1": -73.0812,
"cord2": 40.8732,
"attributes": ["Live Music", "Rooftop Bar", "2 Floors"],
"hours": [
{
"days": "Monday - Friday",
"hours": "7:00am - 7:00pm",
"closed": false
},
{
"days": "Saturday",
"hours": "8:00am - 5:00pm",
"closed": false
},
{
"days": "Sunday",
"closed": true
}
],
"reviews": [
{
"rating": 5,
"id": ObjectId(),
"author": "Jacob G.",
"timestamp": "new Date('Feb 3, 2016')",
"body": "Whoa! The music here is wicked good. Definitely going again."
},
{
"rating": 4,
"id": ObjectId(),
"author": "Tom B.",
"timestamp": "new Date('Feb 23, 2016')",
"body": "I asked to play her a tune. She said no."
}
]
}
}
Additionally, I'm not entirely sure how to specify subdocuments within the JSON (assuming I can get the seeding process to work correctly in the first place).
Here's my model:
var mongoose = require('mongoose');
var subHoursSchema = new mongoose.Schema({
days: {type: String, required: true},
opening: String,
closing: String,
closed: {type: Boolean, required: true}
});
var subReviewsSchema = new mongoose.Schema({
rating: {type: Number, required: true, min: 0, max: 5},
author: String,
timestamp: {type: Date, "default": Date.now},
body: String
});
var locationSchema = new mongoose.Schema({
name: {type: String, required: true},
address: String,
rating: {type: Number, "default": 0, min: 0, max: 5},
attributes: [String],
coordinates: {type: [Number], index: '2dsphere'},
openHours: [subHoursSchema],
reviews: [subReviewsSchema]
});
mongoose.model('Location', locationSchema);
Any insight on how to navigate these issues would be greatly appreciated. Thanks!
You can populate MongoDB in the CLI using mongoimport
It will load a JSON file into a specified MongoDB Instance & Collection, all you need is for a mongod instance to be running before executing.
Here is a walkthrough of using mongoimport.
You JSON is not flowing your schema.
Fix your JSON to this:
{
{
"name": "Dan's Place",
"rating": 3,
"address": "125 High Street, New York, 10001",
"coordinates": [-73.0812, 40.8732],
"attributes": ["Hot drinks", "Food", "Premium wifi"],
"openHours": [
{
"days": "Monday - Friday",
"opening": "7:00am",
"closing": "7:00pm",
"closed": false
},
{
"days": "Saturday",
"opening": "8:00am",
"closing": "5:00pm",
"closed": false
},
{
"days": "Sunday",
"closed": true
}
],
"reviews": [
{
"rating": 4,
"author": "Philly B.",
"timestamp": "new Date('Feb 3, 2016')",
"body": "It was fine, but coffee was a bit dull. Nice atmosphere."
},
{
"rating": 3,
"author": "Tom B.",
"timestamp": "new Date('Feb 23, 2016')",
"body": "I asked for her number. She said no."
}
]
},
{
"name": "Jared's Jive",
"rating": 5,
"address": "747 Fly Court, New York, 10001",
"coordinates": [-73.0812, 40.8732],
"attributes": ["Live Music", "Rooftop Bar", "2 Floors"],
"openHours": [
{
"days": "Monday - Friday",
"opening": "7:00am",
"closing": "7:00pm",
"closed": false
},
{
"days": "Saturday",
"opening": "8:00am",
"closing": "5:00pm",
"closed": false
},
{
"days": "Sunday",
"closed": true
}
],
"reviews": [
{
"rating": 5,
"author": "Jacob G.",
"timestamp": "new Date('Feb 3, 2016')",
"body": "Whoa! The music here is wicked good. Definitely going again."
},
{
"rating": 4,
"author": "Tom B.",
"timestamp": "new Date('Feb 23, 2016')",
"body": "I asked to play her a tune. She said no."
}
]
}
}
You can use mongoose-data-seed to write your own seed script that interacting your mongoose models with:
https://github.com/sharvit/mongoose-data-seed
I solved this issue on a project by dumping the relevant data to an extended JSON array formatted file using mongoexport --jsonArray, then importing this back into POJO format inside the Node application using the EJSON package. I then just use Mongoose to insert the resulting JS array back into the database using the correct collection model you've created using Mongoose.
The necessary JSON data files to seed the application for a first-run are checked into the application repository.
Here's a quick sample you may be able to adapt to your purposes:
// ...
// 'Items' is the Mongoose collection model.
const itemResult = await Items.find({}).exec();
if(itemResult.length === 0) {
const itemsSeedDataRaw = fs.readFileSync(`${__dirname}/data/items.json`, 'utf8');
const itemsSeedData = EJSON.parse(itemsSeedDataRaw);
await Items.insertMany(itemsSeedData);
}
// ...
I would also recommend looking into mongo-seeding. There is both a JS library version and a CLI version. The motivation for this library is described here.

JSON object extract from big query database using app script

I have a big query table that has a JSON object as one of the field in the table. How can I extract the data from the JSON object using app script. The object itself is nested. It looks like this
{
"uid": "124551",
"subjects": [
{
"tid": 37,
"title": "Algebra",
"html_id": "algebra",
"selected": true
},
{
"tid": 214853,
"title": "Trigonometry",
"html_id": "trigonometry",
"selected": true
},
{
"tid": 38,
"title": "Geometry",
"html_id": "geometry",
"selected": true
}
],
"cellphone": "09178854579",
"educations": [
{
"index": 0,
"schoolname": "University of the Philippines - Los Baños",
"degree": "BS Mathematics",
"major": "Mathematics",
"eduFrom": "2009-05-31T16:00:00.000Z",
"eduTo": "2013-04-26T16:00:00.000Z",
"eduFromTs": 1243785600,
"eduToTs": 1366992000
}
],
"info": {
"os": "Windows",
"internet": "ADSL",
"browser": "Chrome",
"network": "Wireless",
"speed": "",
"timezone": "Asia/Hong_Kong"
}
}
I want to extract all school names from education field. Any ideas?
Working with JSON objects is similar to working with XML, except that parsing or encoding a JSON object is much easier.
Once this string is retrieved, simply call JSON.parse() on the string to get a native object representation.
var data = JSON.parse(json);
Browser.msgBox(data.info.os);
Other sample code is at https://developers.google.com/apps-script/advanced/bigquery#run_query

Error getting JSON Map<String, Object> with SpringMVC and Jackson2

I'm trying to get a Map where there are a 'number' with the number of actions and 'actions' with a list of actions.
I've checked the list of actions in java and it works just fine.
I have 4 actions and in json file I'm getting right just the first one.
Controller in java
#RequestMapping(value = "/getactions/{idTask}", method = RequestMethod.GET, produces = "application/json")
#ResponseBody
public Map<String, Object> getActions(Principal principal,
#PathVariable Long idTask) {
logger.info("Task controller get actions...");
List<TaskAction> actions = null;
if (principal == null) {
actions = new ArrayList<TaskAction>();
} else {
actions = taskActionService.getAllTaskActions(idTask);
}
System.out.println(">>>>>>>>Task Controller - Actions>>>>>>>>>>>>> "
+ actions.size());
for (TaskAction ta : actions) {
System.out.println(">>>>>>>>ta: " + ta.getActionname());
}
System.out.println(">>>>>>>>END getAllActions()>>>>>>>>>>>>> ");
Map<String, Object> data = new HashMap<String, Object>();
data.put("actions", actions);
data.put("number", actions.size());
return data;
}
Sysout
>>>>>>>>Task Controller - Actions>>>>>>>>>>>>> 4
>>>>>>>>ta: tacometer
>>>>>>>>ta: hola
>>>>>>>>ta: hi there
>>>>>>>>ta: other action
>>>>>>>>END getAllActions()>>>>>>>>>>>>>
I'm not getting any error, but the json result only gets the first element in the action list and the id of the rest of actions.
json that I get from http...getactions/8
{
"number":4,
"actions":[
{ ... },
3,
4,
5
]
}
The ... is the first action that is well recovered. (I've avoided to write the code for clearity)
Any ideas about what could it be?
Thanks in advance.
Example of json with 2 actions
{
"number": 2,
"actions": [{
"idTaskAction": 4,
"task": {
"idTask": 8,
"taskname": "abbbbbbbbbbbbbb",
"description": "fffffffffffffffqqqqq",
"date": 1389569940000,
"deadline": -23918633280000,
"category": {
"idTaskCategory": 1,
"sortOrder": 0,
"categoryname": "cat1",
"timestamp": 1402437394000
},
"priority": {
"idTaskPriority": 1,
"sortOrder": 0,
"aka": "none",
"priorityname": "low",
"timestamp": 1402437527000
},
"state": {
"idTaskState": 1,
"statename": "pending"
},
"user": {
"idUser": 1,
"username": "joe"
},
"userResponsible": {
"idUser": 1,
"username": "joe"
},
"evaluation": "12345678saad",
"pending": 0,
"actions": [{
"idTaskAction": 2,
"task": 8,
"date": 1402652358000,
"actionname": "tacometer",
"description": "asfdafa",
"duration": 12,
"user": {
"idUser": 1,
"username": "joe"
},
"timestamp": 1411493866000
},
{
"idTaskAction": 3,
"task": 8,
"date": 1404207558000,
"actionname": "hola",
"description": "un dos tres",
"duration": 20,
"user": {
"idUser": 1,
"username": "joe"
},
"timestamp": 1405022827000
},
4,
{
"idTaskAction": 5,
"task": 8,
"date": 1412164741000,
"actionname": "other action",
"description": "ya me my",
"duration": 22,
"user": {
"idUser": 1,
"username": "joe"
},
"timestamp": 1411733131000
}],
"timestamp": 1411733131000
},
"date": 1412074440000,
"actionname": "hi there",
"description": "lkjñlkj ya",
"duration": 25,
"user": {
"idUser": 1,
"username": "joe"
},
"timestamp": 1411733090000
},
5]
}
The problem was that the classes Task, User... has #JsonIdentityInfo with ObjectIdGenerator.PropertyGenerator.class option so when Jackson try to serialize the object into json, if the object it is already in the file, then it is not added again. This way avoid to serialize json file with infinite recursions.
To solve this we need to change ObjectIdGenerator.PropertyGenerator.class for ObjectIdGenerator.None.class so that the objects are serialized even if they are already part of other objects.
Change
#JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "#id")
for
#JsonIdentityInfo(generator = ObjectIdGenerators.None.class, property = "#id")