Insert Schema into MongoDB with HTTP post - json

I need to insert new Schema into MongoDB.The only point of access to the database that I have is through http.post request.
I am currently trying to do it with POSTMAN, but I cannot figure out the syntax I need to send in order to create a schema. This is what I have if someone has ideas I would be grateful for your input.
var TaskSchema = new mongoose.Schema({
title:{type:String,required:true},
instructions:{type:String,required:true},
repeatWeekDay:{type:Number,required:false},
medication:[medicationSchema],
reading:[{
readingType:{type:String,required:true},
measureType:{type:String,required:true},
measureValue:{type:String,required:true},
measureUnits:{type:String,required:true},
measureFormat:{type:String,required:true}
}],
alerts:[{type:Schema.Types.ObjectId,ref:'Alert'}],
plans:[{type:Schema.Types.ObjectId,ref:'Plan'}],
createdAt:{type: Date, default: Date.now },
createdBy:{type:Schema.Types.ObjectId,ref:'User'},
updatedAt:{type: Date, default: Date.now },
updatedBy:{type:Schema.Types.ObjectId,ref:'User'}
});

I think you're laboring under a misapprehension.
mongoose is a JavaScript library/module that enforces structured interaction with the mongodb datastore using Schemas, Types, etc. where a Schema object simply provides necessary programmatic intelligence to its validation function, and is not a data format itself. In other words, internally a mongoose Schema is a function not a data interchange format.
mongodb is a schema-less data storage engine that gleefully accepts any properly formatted JSON document for storage.
I can imagine very few reasons to store a data schema in mongodb.

Related

How do you efficiently handle highly dynamic JSON structures in Swift?

I am writing a Swift program to interact with our backend Database. These database tables can change format, and each row is from DynamoDB and is represented by a JSON object. Within the program, it will grab the cache from the server which contains the Database structures and then the current contents of the database. I don't want to hardcode the database structures into the app as that would obviously make this too very hard to maintain. What is the best way to read in these rows into a generic dictionary object with dynamic names that can hold any of the JSON types?
One way you can handle parsing dynamic JSON is to have a key or a set of keys that gives information about which type of model to decode. For example, you can have a key type in your JSON objects:
{
...
"type": "some type"
...
}
and an intermediate model:
struct IntermediateModel: Codable {
let type: String // or enum
}
you first decode all your JSON objects to the intermediate model and then decode them to the actual model type based on the value of type.
I have written some helper protocols that make this easier here.

MongoDB Realm convert BJSON to JSON

I am using MongoDB Realm Webhooks to allow for my react app to fetch data from my MongoDB Database. Everything works fine, however the data that I receive is not raw JSON, in the sense that every integer field in the object has an additional property which shows its data type. To better clarify this, I will show an example of the data that is returned:
stats: {
draws: {$numberInt: "1"}
games: {$numberInt: "271"}
goals: {$numberInt: "417"}
losses: {$numberInt: "23"}
}
Is there a way where I am able to parse the data so it can be formatted without the datatype? For example:
stats: {
draws: 1,
games: 271,
goals: 417,
losses: 23,
}
This would improve code readability for my frontend. I've tried to flatten the object manually using an object flatten function, but what I am dealing with is a large nested object, so it is difficult.
Try BSON.Binary.fromText or similar functions. There's not a lot of documentation of the BSON utilities

Explicitly providing schema in a form of a json in Spark / Mongodb integration

When integrating spark and mongodb, it is possible to provide a sample schema in a form of an object - as described here: https://docs.mongodb.com/spark-connector/master/scala/datasets-and-sql/#sql-declare-schema
As a short-cut, there is a sample code how one can provide mongodb spark connector with sample schema:
case class Character(name: String, age: Int)
val explicitDF = MongoSpark.load[Character](sparkSession)
explicitDF.printSchema()
I have a collection, which has a constant document structure. I can provide a sample json, however to create a sample object manually will be impossible (30k properties in a document, 1.5MB average size). Is there a way how spark would infer schema just from that very json and would circumvent Mongodb connector's initial sampling which is quite exhaustive?
Spark is able to infer the schema, especially from sources having it as MongoDB. For instance for RDBMS it executes a simple query returning nothing but table columns with their types (SELECT * FROM $table WHERE 1=0).
For the sampling it'll read all documents unless you specify the configuration option called samplingRatio like this:
sparkSession.read.option("samplingRatio", 0.1)
For above Spark will only read 10% of the data. You can of course set any value you want. But be careful because if your documents have inconsistent schemas (e.g. 50% have a field called "A", the others not), the schema deduced by Spark may be incomplete and at the end you can miss some data.
Some time ago I wrote a post about schema projection if you're interested: http://www.waitingforcode.com/apache-spark-sql/schema-projection/read

MongoDB not returning a proper JSON

When I run db.collection.explain().find(), it gives the following error;
The last field in this json object has a double quote problem: `"totalChildMillis" : NumberLong(2)`.
When I parse this object, I got an exception saying that NumberLong(2) should be double quoted. Is there a way for MongoDB returns a standard JSON object?
{
"executionStages":{
"stage": "SINGLE_SHARD",
"nReturned": 10000,
"executionTimeMillis": 3,
"totalKeysExamined": 0,
"totalDocsExamined": 10000,
"totalChildMillis": NumberLong(2)
}
}
EDIT1
I am currently using Javascript NodeJS to create a sub-process of a mongo-shell. And send explain command to that process and listen on its output. Once I got the output, I need to parse it to a javascript object by JSON.parse() method. Based on this use case, what is the easier way for me to adapt mongo json extension to be a standard javascript object?
See the docs on MongoDB Extended JSON. Basically it comes down to the fact that MongoDB extends JSON to add additional datatypes that JSON does not support. In order to preserve that type information, various tools use either "strict" mode (which confirms to the JSON RFC) or "mongo shell" mode, which uses notations like NumberLong() and ISODate() to represent datatypes and is generally not parseable using a JSON parser.
Depending on what you're doing, you can use mongoexport which has an option to output in strict mode. But if you're trying to evaluate the explain plan of a query, I don't think that's going to work unless you insert the explain plan into a temp collection and then mongoexport it out.
Your best bet is to do whatever scripting you're trying to accomplish using a programming language (e.g. Java, Perl, Python, C#, etc.) and one of the corresponding MongoDB drivers. There you'll have much more flexibility and power with how you retrieve and parse data.
Since you mentioned in your edit that you're using Node.js, you can use the explain option to get the explain output directly from Node without having to spawn a sub-process.
Here's a very basic example:
var url = 'mongodb://localhost:27017/test';
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect(url, function(err, db) {
assert.equal(null, err);
var collection = db.collection('test');
collection.find({}, {explain:true}).each(function(err, doc) {
if(doc != null)
console.dir(doc);
});
});

persisting json array data

I've been working with AngularJS and JSON for a while now, and I am currently writing a simple todo app that uses the following array to store its todos:
$scope.todos = [
// todo 1
{
title: 'Personal',
status: 'todo',
// categories for todo 1
categories: [
{
title: 'Shopping',
status: 'doing',
// items for category 1, todo 1
items: [
{
title: 'Buy bacon',
status: 'complete',
},
{
title: 'Buy tuna',
status: 'doing',
},
], // / items
},
], // /categories
},
]; // todos
So far, so well. Now what I am not sure about is how to actually store this data permanently. If I use my application to add or modify a todo, it's all nice and good until I close the browser window and it's all back to the default values (obviously).
Until now, I have always been working with MySQL databases to store relational data. But I was wondering if there is a better way to store this json data?
I was thinking to create a simple php page with saves the whole array to a textfile. But that would mean rewriting the whole file every time I make even the tiniest change to the data.
I've heard there were databases available that allow you to store this type of data, but I don't know where to start? Any pointer would be much appreciated.
Nothing keeps you from saving this in a relation database like MySQL, you could have entities like a Todo, Category and Item, then serialize then into JSON and serve them RESTfully.
I think what you were looking for is a NoSQL database. They can store JSON data natively and can store chunks of data instead of just rows of data like traditional relational databases.
Two popular NoSQL databases are
MongoDB
RethinkDB
I would suggest going with a framework like restangular to define your relations, you will then be able to use all kinds of noSQL databases which have a RESTfull JSON API such as couchdb or mongodb etc.
It uses promises which is nice future proof and modern, it also supports all HTTP methods you might need, but it has a lot more features than that, take a look at the repo's readme.
Here is also a demo which uses mongolabs, mongodb flawored cloud service.
Hope it helps.