I am learning MongoDB. The thing that is confusing me right now is collections don't enforce the strict structure like RDBMS. I created a document with following parameters:
db.tempTable.insert({
id: 1,
name: umer
});
then I inserted:
db.tempTable.insert({
"id":2,
"name": "khan",
"yolo":"todolo"
});
Both the queries were successful. I thought Mongo would raise an exception in the second insertion because the collection didn;t have the "YOLO" column in the collection.
Why is that? Can we enforce the strict structure to avoid confusions while inserting in the table?
Best
No, MongoDB is schemaless. This means there is no strict schema. The only way you can raise an exception to ensure all documents have the same fields is at your code level, perhaps by using ORM
Like #hd says, mongodb is schemaless. But there is solutions to play with like validators that allow you to validate data before to insert them.
#example of new validator
db.runCommand( {
collMod: "contacts",
validator: { $or: [ { phone: { $exists: true } }, { email: { $exists: true } } ] },
validationLevel: "moderate"
} )
Related
This issue is a bit tricky to describe so bear with me and please ask questions if I am missing anything...
Say you have a json object that defines a list of features, each feature has a the same three properties but has a property that has an entirely different structure. For example:
{
features: [
{
id: "feature-a",
enabled: true,
configurationData: {
featureAConfigPropertyA: {
somePrperty: "whatever",
anotherProperty: true
},
featureAConfigPropertyB: "some string"
}
},
{
id: "feature-b",
enabled: true,
configurationData: {
featureBConfigArrayPropertyA: ["some string"],
featureBConfigPropertyB: [
{
"id": "some string",
"name": "some string",
"description": "some string",
"enabled": true
}
]
}
}
]
}
The actual structure of each feature is irrelevant. I am just trying to express this via json schema whereby the structure of configurationData for each feature is dependent on or dictated by the feature id value of its parent.
EDIT: I guess technically it doesnt need to be dependent on so long as either structure of configurationData is valid schema for that property on the feature schema itself. Also, the types in configurationData arent arbitrary, they would always be one of the two types for a given feature in this example.
This however needs to be structured in a way that can be expressed via Formly as I am using this to generate forms. In this case it would be an array of ObjectFieldTypes, one for feature a and one for feature b, which would enumerate the three properties and provide Input field types, until it got to configurationData at which point it would use an ObjectFieldType again, which would now be different for each field type.
The issue here is that 1) I'm not sure how to express this in json schema and 2) I can't use things like patternProperties with formly because the properties have to be explicitly defined in the json schema in order for formly to render the field types for each property. Although patternProperties would technically be valid schema in this case, if the schema doesn't define those properties, then the model in the valueChanges observable on the FormGroup just excludes them entirely. So I would end up with:
{
features:[
{
id: "feature-a",
enabled: true,
configurationData: { }
},
{
id: "feature-b",
enabled: true,
configurationData: { }
}
]
}
I have tried the if then else construct, but I cant tell if the schema is wrong or if formly just doesn't support this. I made a stack blitz for this below:
https://stackblitz.com/edit/angular-g45ydm?file=src%2Fassets%2Fjson-schema%2Fif_then.json
I have an database full of objects that contain information, as well as other arrays of objects. I would like to change the inner arrays to only be arrays with each index as an ObjectId type with their respective ObjectId
I am using the mongoose populate function to retrieve this information later in the program. So only the ObjectId is needed for the reference.
job {
_id: 1,
name: "name",
parts: [
{ _id: ObjectId("5c790ce7d3dc8d00ccc2688c"), name: "name"},
{ _id: ObjectId("5c790ce7d3dc8d00ccc2688b"), name: "name"},
{ _id: ObjectId("5c790ce7d3dc8d00ccc2688a"), name: "name"},
]
}
Desired Result
job {
_id: 1,
name: "name",
parts: [
ObjectId("5c790ce7d3dc8d00ccc2688c"),
ObjectId("5c790ce7d3dc8d00ccc2688b"),
ObjectId("5c790ce7d3dc8d00ccc2688a")
]
}
I tried a few mongoDB queries from the command line but none of them are quite giving the result I need. This one has no errors, but it doesn't seem to change anything.
db.jobs.update(
{},
{
$set: {"parts.$[element]": "element._id"}
},
{
multi: true,
arrayFilters: [{ "element": {} }]
}
)
I'm not sure if this is possible or not using only the mongo shell.
Mongo v4.2 introduced pipelined updates this allows the use of aggregation operators within the update and more importantly updating a document using it's own values. which is what you want to achieve.
db.jobs.updateMany(
{},
[
{
'$set': {
'parts': {
'$map': {
'input': '$parts',
'as': 'part',
'in': '$$part._id'
}
}
}
}
]);
Unfortunately this is not possible for earlier Mongo versions. Specifically $arrayFilters allows you to filter an array but again prior to the new update syntax accessing own values within the update is not possible.
You'll have to iterate over all documents and do the update one by one in code.
As #Tom Slabbaert mentioned in the other answer you will have to use updates with aggregation pipelines available from v4.2 if you want to update the documents in bulk in one operation.
As an alternative to using $map, if you want only a single value at the top level and the value is accessible using the dot notation. You can simply use $set with the dot notation.
db.jobs.updateMany({}, [{
$set: { parts: "$parts._id" }
}])
Trying to figuring out how to deserialize this kind of json in talend components :
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
"ryan.buckley#toofr.com": {
"confidence":18,"email":"ryan.buckley#toofr.com","default":16
},
"ryanbuckley#toofr.com": {
"confidence":17,"email":"ryanbuckley#toofr.com","default":17
},
"ryan_buckley#toofr.com": {
"confidence":16,"email":"ryan_buckley#toofr.com","default":18
},
"ryan-buckley#toofr.com": {
"confidence":15,"email":"ryan-buckley#toofr.com","default":19
},
"ryanb#toofr.com": {
"confidence":14,"email":"ryanb#toofr.com","default":14
},
"buckley#toofr.com": {
"confidence":13,"email":"buckley#toofr.com","default":13
}
}
This JSON comes from the Toofr API where documentation can be found here .
Here the actual sitation :
For each line retreived in the database, I call the API and I got this (the first name, the last name and the company change everytime.
Does anyone know how to modify the tExtractJSONField (or use smthing else) to show the results in tLogRow (for each line in the database) ?
Thank you in advance !
EDIT 1:
Here's my tExtractJSONfields :
When using tExtractJSONFields with XPath, you need
1) a valid XPath loop point
2) valid XPath mapping to your structure relative to the loop path
Also, when using XPath with Talend, every value needs a key. The key cannot change if you want to loop over it. Meaning this is invalid:
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
but this structure would be valid:
{
"contact": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"contact": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
So with the correct data the loop point might be /contact.
Then the mapping for Confidence would be confidence (the name from the JSON), the mapping for Email would be email and vice versa for default.
EDIT
JSONPath has a few disadvantages, one of them being you cannot go higher up in the hierarchy. You can try finding out the correct query with jsonpath.com
The loop expression could be $.*. I am not sure if that will satisfy your need, though - it has been a while since I've been using JSONPath in Talend because of the downsides.
I have been ingesting some complex json structures and did this via minimal json libraries, and tjava components within talend.
I want to create a Tree like storage structure to be used with my app, but can't find enough documentation on how to create a tree model using Waterline attributes.
Case is simple. I do need to have a set of folders, that can have multiple levels of subfolders and in the end, files. What you usually do in mySQL for this kind of data is to add a parent_id field to your model as a foreign key to the model itself.
How can this be done using attributes in sailsjs/waterline model?
I've tried doing something like the following, which generates quite a bit of redundant and orphaned data:
--
attributes: {
name: {
type: 'string'
},
parentFolder: {
model: 'Folder'
},
childFolders: {
model: 'Folder',
via: 'parentItem'
}
}
--
Any ideas?
And by the way, if this is possible, let's say for example using mySQL as a backend. How will it replicate to say, mongoDB?
This seemed to work:
name: {
type: 'string',
maxLength: 255,
required: true
},
parent: {
model: 'folder'
},
childs: {
collection: 'folder',
via: 'parent'
}
I do believe duplicates were being generated by posting data directly via GET in the browser. I'm posting data with a client via POST and it seems to work as expected. (At least from what I see in mySQL)
I am new to sails.js and mongodb. I found something strange. When i use rest api to update the record in mongodb, after record updated, the json format changed.
For example. Originally I have a record like this:
{
creator: "John",
taskname: "test",
id: "53281a5d709602dc17b000cd"
}
After clicking http://127.xxx:1337/testtask/update/53281a5d709602dc17b000cd?creator=default%20creator, following json returned.
The json field is sorted in alphabetic order.
How can i keep the origin format of the json file? Is it a bug? Is there any workaround?
{
createdAt: "2014-03-18T10:05:17.052Z",
creator: "default creator",
taskname: "test",
updatedAt: "2014-03-18T10:08:53.067Z",
id: "53281a5d709602dc17b000cd"
}
Thanks.
The problem is fields in JSON objects don't have any concept of order. A JSON object is a dictionary, or in other words just some key/value pairs. This means that this JSON:
{ "a" : "some string", "b" : "other string" }
is logically equivalent to this JSON:
{ "b" : "other string", "a" : "some string" }
If you want to preserve ordering in your JSON data there are other ways to do it. For example JSON arrays do preserve order so something like this would work:
[ { "a" : "some string" }, { "b" : "other string" } ]
Internally MongoDB may actually preserve the ordering, but that's an implementation detail and you can't depend on it.
More detail on what Mongo is doing here.
Much like the "other" framework that inspired this, there is some automatic time-stamp generation happening in your models when things are updated or created. You wouldn't be the first. Ruby people have been trapped by this for years.
There are options you can define on your collection objects to remove these fields. This comes from the Waterline documentation, which should be the manager in use.
So in addition to attributes:
autoCreatedAt: false,
autoUpdatedAt: false,
attributes: {
// normal things here
},
Of course you will need to remove any of these properties that have been created in your documents manually. See the $unset operator for doing this