Mongoose Model for Nested Array JSON - json

I need to create a Mongoose model for the below nested JSON array. The issue I am facing is TLSM01 is a dynamic Key and I am unable to specify it in the model. If I mention entities alone and pass all the json objects as string it is storing as [object] and not data.
"entities": [
{
"TLSM01": [
{
"01": {
"Name": "Light",
"Properties": [
{
"state": [
{
"type": "boolean",
"propertyMode": "actuator"
}
],
"brightness": [
{
"type": "integer",
"propertyMode": "actuator"
}
]
}
]
}
}
]
}
Mongoose Model:
var thingsSchema = ({
"uuid": String,
"things": String,
"manufacturerName": String,
"manufacturerId": String,
"osName": String,
"hardwareVersion": String,
"firmwareVersion": String,
"entity": [{String}]
})

Store data in key-value pair
"entities": [
{
keyName:'TLSM01',
data: [
{
"01": {
"Name": "Light",
"Properties": [
{
"state": [
{
"type": "boolean",
"propertyMode": "actuator"
}
],
"brightness": [
{
"type": "integer",
"propertyMode": "actuator"
}
]
}
]
}
}
]
}
]
Mongoose Model:
var thingsSchema = ({
"uuid": String,
"things": String,
"manufacturerName": String,
"manufacturerId": String,
"osName": String,
"hardwareVersion": String,
"firmwareVersion": String,
"entity": [{_id:false,keyName:{type:String},data:[]}]
})

1.I think you need something structured like this:
2.See how the light value is an array...within the object - must use key value paring in Mongo and ensure you model Json Object can be mapped to you mongoose database - hence you use you data model to input data - so this should work if you augment it..
const blogSchema = new Schema({
name:{
type:String,
require: true
},
heat:{
type:Number,
Require:true
},
moisture:{
type:Number,
Require:true
},
light:{
green:{type:Number, Require:true},
red:{type:Number, Require:true},
blue:{type:Number, Require:true},
white:{type:Number, Require:true}
},
body:{
type:String,
require: true
}
},{timeStamps:true});

Related

RealmList<String> as a JSON Schema - Mongo DB Realm

I have a simple model class from which I need to generate the schema on Mongo DB Atlas. But I'm having troubles when it comes to defining RealmList<String> inside a JSON schema. If I insert "array" as a bsonType, I get an error. What should I write instead?
Model class:
class Note : RealmObject {
#PrimaryKey
var _id: ObjectId = ObjectId.create()
var title: String = ""
var description: String = ""
var images: RealmList<String> = realmListOf()
var date: RealmInstant = RealmInstant.from(System.currentTimeMillis(),0)
}
Current Schema:
{
"bsonType": "object",
"properties": {
"_id": {
"bsonType": "objectId"
},
"title": {
"bsonType": "string"
},
"description": {
"bsonType": "string"
},
"images": {
"bsonType": "array"
},
"date": {
"bsonType": "date"
}
},
"required": [
"_id",
"title",
"description",
"images",
"date"
],
"title": "Note"
}
I am not sure which mode you're using but if you're in development mode, when you add an object in the SDK, the server will automatically generate a matching object, as long as the changes are additive, like adding a new object or property
In the queston, the 'images' bson definition looks incomplete
"images": {
"bsonType": "array"
},
While it is an array, it's an array of strings so I believe it should look more like this
"images": {
"bsonType": "array",
"items": {
"bsonType": "string"
}
}
Where the type of items is defined as a string

Parse Complex JSON -- Map

I need to parse the complex JSON (below) IN SCALA to get the values of "expression" and "value" in "measure" key i.e I need List (COUNT, COUNT_DISTINCT ...) and List (1,tbl1.USER_ID ...).
I tried multiple options, but it is not working. Any help is appreciated
{
"uuid": "uuidddd",
"last_modified": 1559080222953,
"version": "2.6.1.0",
"name": "FULL_DAY_2_mand_date",
"is_draft": false,
"model_name": "FULL_DAY_1_may05",
"description": "",
"null_string": null,
"dimensions": [
{
"name": "PLATFORM",
"table": "tbl1",
"column": "PLATFORM",
"derived": null
},
{
"name": "OS_VERSION",
"table": "tbl1",
"column": "OS_VERSION",
"derived": null
}
],
"measures": [
{
"name": "_COUNT_",
"function": {
"expression": "COUNT",
"parameter": {
"type": "constant",
"value": "1"
},
"returntype": "bigint"
}
},
{
"name": "UU",
"function": {
"expression": "COUNT_DISTINCT",
"parameter": {
"type": "column",
"value": "tbl1.USER_ID"
},
"returntype": "hllc(12)"
}
},
{
"name": "CONT_SIZE",
"function": {
"expression": "SUM",
"parameter": {
"type": "column",
"value": "tbl1.SIZE"
},
"returntype": "bigint"
}
},
{
"name": "CONT_COUNT",
"function": {
"expression": "SUM",
"parameter": {
"type": "column",
"value": "tbl1.COUNT"
},
"returntype": "bigint"
}
}
],
"dictionaries": [],
"rowkey": {
"rowkey_columns": [
{
"column": "tbl1.OS_VERSION",
"encoding": "dict",
"encoding_version": 1,
"isShardBy": false
},
{
"column": "tbl1.PLATFORM",
"encoding": "dict",
"encoding_version": 1,
"isShardBy": false
},
{
"column": "tbl1.DEVICE_FAMILY",
"encoding": "dict",
"encoding_version": 1,
"isShardBy": false
}
]
},
"hbase_mapping": {
"column_family": [
{
"name": "F1",
"columns": [
{
"qualifier": "M",
"measure_refs": [
"_COUNT_",
"CONT_SIZE",
"CONT_COUNT"
]
}
]
},
{
"name": "F2",
"columns": [
{
"qualifier": "M",
"measure_refs": [
"UU"
]
}
]
}
]
},
"aggregation_groups": [
{
"includes": [
"tbl1.PLATFORM",
"tbl1.OS_VERSION"
],
"select_rule": {
"hierarchy_dims": [],
"mandatory_dims": [
"tbl1.DATE_HR"
],
"joint_dims": []
}
}
],
"signature": "ttrrs==",
"notify_list": [],
"status_need_notify": [
"ERROR",
"DISCARDED",
"SUCCEED"
],
"partition_date_start": 0,
"partition_date_end": 3153600000000,
"auto_merge_time_ranges": [
604800000,
2419200000
],
"volatile_range": 0,
"retention_range": 0,
"engine_type": 4,
"storage_type": 2,
"override_kylin_properties": {
"job.queuename": "root.production.P0",
"is-mandatory-only-valid": "true"
},
"cuboid_black_list": [],
"parent_forward": 3,
"mandatory_dimension_set_list": [],
"snapshot_table_desc_list": []
}
This is a snippet of the code I tried, and it is giving a null list
import org.json4s._
import org.json4s.jackson.JsonMethods._
implicit val formats = org.json4s.DefaultFormats
case class Function (
expression: String,
parameter: Parameter,
returntype: String
)
case class Parameter (
`type`: String,
value: String
)
case class Measures (
name: String,
function: Function
)
case class AllMeasuresData(uuid: String, measure: List[Measures])
val data = parse(tmp).extract[AllMeasuresData]
val names = data.measure.map(_.name)
println(names)
case class AllMeasuresData(uuid: String, measure: List[Measures])
val data = parse(tmp).extract[AllMeasuresData]
val names = data.measure.map(_.name)
println(names)
There are couple typos in your ADT:
Here is what you need:
case class Function (
expression: String,
parameter: Parameter,
returntype: String
)
case class Parameter (
`type`: String,
value: String
)
case class Measures (
name: String,
function: Function
)
case class AllMeasuresData(uuid: String, measures: List[Measures])
There is also an extra comma int the json, here is the correct:
{
"uuid":"uuidddd",
"last_modified":1559080222953,
"version":"2.6.1.0",
"name":"FULL_DAY_2_mand_date",
"is_draft":false,
"model_name":"FULL_DAY_1_may05",
"description":"",
"null_string":null,
"dimensions":[
{
"name":"PLATFORM",
"table":"tbl1",
"column":"PLATFORM",
"derived":null
},
{
"name":"OS_VERSION",
"table":"tbl1",
"column":"OS_VERSION",
"derived":null
} // There was an extra trailing comma here
],
"measures":[
{
"name":"_COUNT_",
"function":{
"expression":"COUNT",
"parameter":{
"type":"constant",
"value":"1"
},
"returntype":"bigint"
}
},
{
"name":"UU",
"function":{
"expression":"COUNT_DISTINCT",
"parameter":{
"type":"column",
"value":"tbl1.USER_ID"
},
"returntype":"hllc(12)"
}
},
{
"name":"CONT_SIZE",
"function":{
"expression":"SUM",
"parameter":{
"type":"column",
"value":"tbl1.SIZE"
},
"returntype":"bigint"
}
},
{
"name":"CONT_COUNT",
"function":{
"expression":"SUM",
"parameter":{
"type":"column",
"value":"tbl1.COUNT"
},
"returntype":"bigint"
}
}
],
"dictionaries":[
],
"rowkey":{
"rowkey_columns":[
{
"column":"tbl1.OS_VERSION",
"encoding":"dict",
"encoding_version":1,
"isShardBy":false
},
{
"column":"tbl1.PLATFORM",
"encoding":"dict",
"encoding_version":1,
"isShardBy":false
},
{
"column":"tbl1.DEVICE_FAMILY",
"encoding":"dict",
"encoding_version":1,
"isShardBy":false
}
]
},
"hbase_mapping":{
"column_family":[
{
"name":"F1",
"columns":[
{
"qualifier":"M",
"measure_refs":[
"_COUNT_",
"CONT_SIZE",
"CONT_COUNT"
]
}
]
},
{
"name":"F2",
"columns":[
{
"qualifier":"M",
"measure_refs":[
"UU"
]
}
]
}
]
},
"aggregation_groups":[
{
"includes":[
"tbl1.PLATFORM",
"tbl1.OS_VERSION"
],
"select_rule":{
"hierarchy_dims":[
],
"mandatory_dims":[
"tbl1.DATE_HR"
],
"joint_dims":[
]
}
}
],
"signature":"ttrrs==",
"notify_list":[
],
"status_need_notify":[
"ERROR",
"DISCARDED",
"SUCCEED"
],
"partition_date_start":0,
"partition_date_end":3153600000000,
"auto_merge_time_ranges":[
604800000,
2419200000
],
"volatile_range":0,
"retention_range":0,
"engine_type":4,
"storage_type":2,
"override_kylin_properties":{
"job.queuename":"root.production.P0",
"is-mandatory-only-valid":"true"
},
"cuboid_black_list":[
],
"parent_forward":3,
"mandatory_dimension_set_list":[
],
"snapshot_table_desc_list":[
]
}
Now you can run:
val data = parse(tmp).extract[AllMeasuresData]
val names = data.measures.map(_.name)
println(names)
// Displays
// List(_COUNT_, UU, CONT_SIZE, CONT_COUNT)
Your Parameter class does not match the JSON because you have used type1 rather than type as the field name. Use backticks to use "type" as a field name even though it is a reserved word:
case class Parameter (
`type`: String,
value: String
)
You also need to change the Function class as it has returntype1 rather than returntype:
case class Function (
expression: String,
parameter: Parameter,
returntype: String
)
The names of the fields in Scala must exactly match the names of the fields in the JSON. Extra fields in the JSON are ignored, but all the fields in the Scala must have matching fields in the JSON. If there are optional fields in the JSON then the Scala field type should be Option[...].

Convert Array to JSON with ID's as Index (MongoDB / Express)

I have an user schema in my mongodb database which saves users. In mongoose I write:
const userSchema = new Schema({
email: { type: String, unique: true, lowercase: true},
password: String,
fname: String,
lname: String,
articles: [{
type: Schema.Types.ObjectId,
ref: "article"
}]
})
I've saved a few users to that Collection. When I query my api for users, I get back an array of User objects. Like so:
[
{
"first_name": "Bob",
"_id": "5b36292760afa11b9a216945",
"email": "user#mail.de",
"__v": 0
},
{
"first_name": "Lisa",
"_id": "5bafkjahf123123123123125",
"email": "lisa#mail.de",
"__v": 0
}
]
Is this normal behaviour? And is this coming from the Mongo side of things or the express side? I heard that something like this is more common:
{
"5b36292760afa11b9a216945":{
"first_name": "Bob",
"email": "user#mail.de",
"__v": 0
},
"5bafkjahf123123123123125":
{
"first_name": "Lisa",
"email": "lisa#mail.de",
"__v": 0
}
}
? How could I convert my array into this kind of a JSON? Is there some functionality maybe within mongo that does that for me?
I don't think you can get an object like that back from mongo, but it's not hard to convert it with Array.reduce().
let returnedValue = [{"first_name": "Bob","_id": "5b36292760afa11b9a216945","email": "user#mail.de","__v": 0},{"first_name": "Lisa","_id": "5bafkjahf123123123123125","email": "lisa#mail.de","__v": 0}]
let result = returnedValue.reduce((a, c) => (a[c._id] = c, a), {})
console.log(result)
This leaves the _id in the original object, but if you needed to inside the reduce function.
You can try below aggregation as well
db.collection.aggregate([
{ "$group": {
"_id": null,
"data": {
"$push": {
"k": "$_id",
"v": {
"first_name": "$first_name",
"email": "$email"
}
}
}
}},
{ "$replaceRoot": {
"newRoot": { "$arrayToObject": "$data" }
}}
])
Output
[
{
"5b36292760afa11b9a216945": {
"email": "user#mail.de",
"first_name": "Bob"
},
"5bafkjahf123123123123125": {
"email": "lisa#mail.de",
"first_name": "Lisa"
}
}
]

Using JSON API Serializer to create more complicated JSON

The examples here don't go nearly far enough in explaining how to produce a more complicated structure...
If I want to end up with something like:
{
"data": {
"type": "mobile_screens",
"id": "1",
"attributes": {
"title": "Watch"
},
"relationships": {
"mobile_screen_components": {
"data": [
{
"id": "1_1",
"type": "mobile_screen_components"
},
{
"id": "1_2",
"type": "mobile_screen_components"
},
...
]
}
}
},
"included": [
{
"id": "1_1",
"type": "mobile_screen_components",
"attributes": {
"title": "Featured Playlist",
"display_type": "shelf"
},
"relationships": {
"playlist": {
"data": {
"id": "938973798001",
"type": "playlists"
}
}
}
},
{
"id": "938973798001",
"type": "playlists",
"relationships": {
"videos": {
"data": [
{
"id": "5536725488001",
"type": "videos"
},
{
"id": "5535943875001",
"type": "videos"
}
]
}
}
},
{
"id": "5536725488001",
"type": "videos",
"attributes": {
"duration": 78321,
"live_stream": false,
"thumbnail": {
"width": 1280,
"url":
"http://xxx.jpg?pubId=694940094001",
"height": 720
},
"last_published_date": "2017-08-09T18:26:04.899Z",
"streams": [
{
"url":
"http://xxx.m3u8",
"mime_type": "MP4"
}
],
"last_modified_date": "2017-08-09T18:26:27.621Z",
"description": "xxx",
"fn__media_tags": [
"weather",
"personality"
],
"created_date": "2017-08-09T18:23:16.830Z",
"title": "NOAA predicts most active hurricane season since 2010",
"fn__tve_authentication_required": false
}
},
...,
]
}
what is the most simple data structure and serializer I can set up?
I get stumped after something like:
const mobile_screen_components = responses.map((currentValue, index) => {
id[`id_${index}`];
});
const dataSet = {
id: 1,
title: 'Watch',
mobile_screen_components,
};
const ScreenSerializer = new JSONAPISerializer('mobile_screens', {
attributes: ['title', 'mobile_screen_components'],
mobile_screen_components: {
ref: 'id',
}
});
Which only gives me:
{
"data": {
"type": "mobile_screens",
"id": "1",
"attributes": { "title": "Watch" },
"relationships": {
"mobile-screen-components": {
"data": [
{ "type": "mobile_screen_components", "id": "1_0" },
{ "type": "mobile_screen_components", "id": "1_1" },
{ "type": "mobile_screen_components", "id": "1_2" },
{ "type": "mobile_screen_components", "id": "1_3" },
{ "type": "mobile_screen_components", "id": "1_4" },
{ "type": "mobile_screen_components", "id": "1_5" }
]
}
}
}
}
I have no idea how to get the "included" sibling to "data." etc.
So, the question is:
what is the most simple data structure and serializer I can set up?
Below is the simplest object that can be converted to JSON similar to JSON in the question using jsonapi-serializer:
let dataSet = {
id: '1',
title: 'Watch',
mobile_screen_components: [
{
id: '1_1',
title: 'Featured Playlists',
display_type: 'shelf',
playlists: {
id: 938973798001,
videos: [
{
id: 5536725488001,
duration: 78321,
live_stream: false
},
{
id: 5535943875001,
duration: 52621,
live_stream: true
}
]
}
}
]
};
To serialize this object to JSON API, I used the following code:
let json = new JSONAPISerializer('mobile_screen', {
attributes: ['id', 'title', 'mobile_screen_components'],
mobile_screen_components: {
ref: 'id',
attributes: ['id', 'title', 'display_type', 'playlists'],
playlists: {
ref: 'id',
attributes: ['id', 'videos'],
videos: {
ref: 'id',
attributes: ['id', 'duration', 'live_stream']
}
}
}
}).serialize(dataSet);
console.log(JSON.stringify(json, null, 2));
The first parameter of JSONAPISerializer constructor is the resource type.
The second parameter is the serialization options.
Each level of the options equals to the level of the nested object in serialized object.
ref - if present, it's considered as a relationships.
attributes - an array of attributes to show.
Introduction
First of all we have to understand the JSON API document data structure
[0.1] Refering to the top level (object root keys) :
A document MUST contain at least one of the following top-level
members:
data: the document’s “primary data”
errors: an array of error objects
meta: a meta object that contains non-standard meta-information.
A document MAY contain any of these top-level members:
jsonapi: an object describing the server’s implementation
links: a links object related to the primary data.
included: an array of resource objects that are related to the primary data and/or each other (“included resources”).
[0.2]
The document’s “primary data” is a representation of the resource or
collection of resources targeted by a request.
Primary data MUST be either:
a single resource identifier object, or
null, for requests that target single resources
an array of resource identifier
objects, or an empty array ([]), for reqs. that target
collections
Example
The following primary data is a single resource object:
{
"data": {
"type": "articles",
"id": "1",
"attributes": {
// ... this article's attributes
},
"relationships": {
// ... this article's relationships
}
}
}
In the (jsonapi-serializer) documentation : Available serialization option (opts argument)
So in order to add the included (top-level member) I performed the following test :
var JsonApiSerializer = require('jsonapi-serializer').Serializer;
const DATASET = {
id:23,title:'Lifestyle',slug:'lifestyle',
subcategories: [
{description:'Practices for becoming 31337.',id:1337,title:'Elite'},
{description:'Practices for health.',id:69,title:'Vitality'}
]
}
const TEMPLATE = {
topLevelLinks:{self:'http://example.com'},
dataLinks:{self:function(collection){return 'http://example.com/'+collection.id}},
attributes:['title','slug','subcategories'],
subcategories:{ref:'id',attributes:['id','title','description']}
}
let SERIALIZER = new JsonApiSerializer('pratices', DATASET, TEMPLATE)
console.log(SERIALIZER)
With the following output :
{ links: { self: 'http://example.com' },
included:
[ { type: 'subcategories', id: '1337', attributes: [Object] },
{ type: 'subcategories', id: '69', attributes: [Object] } ],
data:
{ type: 'pratices',
id: '23',
links: { self: 'http://example.com/23' },
attributes: { title: 'Lifestyle', slug: 'lifestyle' },
relationships: { subcategories: [Object] } } }
As you may observe, the included is correctly populated.
NOTE : If you need more help with your dataSet, edit your question with the original data.

Parse JSON array in Typescript

i have a JSON response from remote server in this way:
{
"string": [
{
"id": 223,
"name": "String",
"sug": "string",
"description": "string",
"jId": 530,
"pcs": [{
"id": 24723,
"name": "String",
"sug": "string"
}]
}, {
"id": 247944,
"name": "String",
"sug": "string",
"description": "string",
"jlId": 531,
"pcs": [{
"id": 24744,
"name": "String",
"sug": "string"
}]
}
]
}
In order to parse the response, to list out the "name" & "description", i have written this code out:
interface MyObj {
name: string
desc: string
}
let obj: MyObj = JSON.parse(data.toString());
My question is how do i obtain the name and description into a list that can be displayed.
You gave incorrect type to your parsed data. Should be something like this:
interface MyObj {
name: string
description: string
}
let obj: { string: MyObj[] } = JSON.parse(data.toString());
So it's not MyObj, it's object with property string containing array of MyObj. Than you can access this data like this:
console.log(obj.string[0].name, obj.string[0].description);
Instead of using anonymous type, you can also define interface for it:
interface MyRootObj {
string: MyObj[];
}
let obj: MyRootObj = JSON.parse(data.toString());