Im trying to understand if dynamodb is the right solution for notifications. Each item in a dynamodb table can be 400kb. What I'm trying to understand is how much is 400kb of json? How many key value pairs would that be? over 10000? More? I just don't have any idea.
Related
I doing a drone project that I want to save drone flight path.
as title, I have a column that needs to save an array coordinate data.
my schema:
I save the data through serialization temporarily, but I' not sure that's a good idea, maybe I should use JSON? Or I should use NoSQL？
Or is there any better way to solve this kind of problem？
I have a dataset in Mysql that contains movies , ratings and users that made these ratings on users. I really struggle to understand how I will construct my dataset from mysql schema in order to fit and get imported to mongo.I have extracted the dataset from mysql in json format so now I need to modify it for example embed the ratings into the movies. I would like to ask for some advice on how to do this. Is there any tool available or I should code something to extract from mysql and construct the json? Sorry for my noob question
Way of storing data by MongoDB is nearly the same as JSON. Mongo doesn't force you to define any data model, so you can do just something like that:
db.yourcollection.insert(JSON.parse(yourJSONFile));
But of course there are many advantages of validating data. For these purposes I recommend SimpleSchema package.
Alright so, I'm taking massive amounts in what is supposed to be JSON format and I'm trying to insert it into a Cassandra cluster. The problem is that the data doesn't have a standard key:value format so I believe its not actually JSON.
Here's an example of the data:
'{"15151162":"6f0aa7ebc60af9b6dd5992341e155138b3ea369a","15149182":"c141929a6ccc6157f4de7055ea565e7a83f59aea","15144225":"f70a2cdecee0e7e9fe85819e74d0e09d36060909"}'
So then keeping that in mind I'm wondering and I know this is somewhat opinion, but do I have to pull apart and then mass insert the data or is there a better way where I could just map them to columns using some feature of CQL/Cassandra.
Also as additional information we're talking around 28 million records so ideally I'd like to do it using the CQL/Cassandra instead of reorganizing the objects in a programming language.
I am familiar with Java, C++ and SQL, fairly new to nosql/Hybrid nosql.
Thanks
If you don't have a key then create one as you ingest it by creating a guid and format the record like so.
{
"key": "3fa55ea6-de8b-4b6f-b11e-5a3701982c65",
"type": "weird data",
"data": {
"15144225": "f70a2cdecee0e7e9fe85819e74d0e09d36060909",
"15149182": "c141929a6ccc6157f4de7055ea565e7a83f59aea",
"15151162": "6f0aa7ebc60af9b6dd5992341e155138b3ea369a"
}
}
Adding a type field is very helpful for when the next programmer actually has to deserialize this data. A version field is probably a good idea too.
I am pretty comfortable with basic MySQL commands, as well as complex joins, but not sure about this array or data type that I have come across.
The code looks like:
a:4:{i:9888;s:0:"";i:17148;s:0:"";i:9879;s:0:"";i:9881;s:0:"";}
a:1:{i:9857;s:0:"";}
a:0:{}
Can someone point me in the right direction of the following:
What is this called?
How can I do some basic commands without PHP to loop through? Such as get everything
where A is greater than 1?
Getting every row that includes 9857.
This is php serialized values: http://php.net/manual/en/function.serialize.php
AFAIK, you cannot directly deserialize it from mysql.
Additionally to the explanation what the string actually is, I'm telling you about an array in MySQL.
In fact, it doesn't exist. You could build a dirty kind of array by using dynamic columns but it's really, really dirty.
There is also SET and ENUM which translate a (predefined-) set of data into a bitmask. I am working on an online shop with several hundreds of tables we didn't use ENUM and SET once.
A proper way of implementing arrays in MySQL is a many-to-many or any "many" related relation between entities.
A Car having several Wheels (an array of wheels!)? Several Wheels attached to a single Car?
You would build two tables and connect them through a foreign key to implement an array. Tho this is not a "datatype" kind of array, but a "relation" array.
I'm having some troubles with designing my website. I'm trying to use OOP design in the way I design my site and using MySQL to store the objects in JSON format.
So I'm creating a MySQL table inside my database. The table is going to contain a primary key (PK) and what I call page type (pageType, ex: homePage, aMessage, aTutorial, etc). This means that I will have serval different pages that have different page formats (pageFormats, ex: headerArea, contentArea, footerArea, etc). So depending on the pageType object that was requested, the query would then go to pageFormat table to retrieve the desired divs.
So for example, we have a AJAX request that says the pageType is set to homePage. The request would then go to the pageFormat table and see which divs the homePage is allowed and then return them. I then of course would write them to the document and continue on loading the page with desired content and so forth.
I am just having trouble going from my UML / documentation to actual development of this idea. So if someone could help me with this it'd be greatly appreciated. The trouble that is most difficult for me to understand is that in MySQL database I have is setting up the table for pageType and pageFormat.
The returned types would be in JSON form so the scripts of my page would be able to format them correctly. So that leads me to my second question of what is the best way to store JSON objects in a MySQL table that are going to be divs? Would it be TINYTEXT? Because I don't plan on having large amount of text in there?
Then my last question would be, what would be the best table type? I'm having trouble with selecting this as well.
I have referenced http://dev.mysql.com/doc/refman/5.1/en/storage-engines.html for the table types to try and help me though I'm still unsure.
I also have been reading http://www.agiledata.org/essays/mappingObjects.html#BasicConcepts to understand how to implement relational databases and mapping objects to them. Is there any other good reads that I should look into?
Thanks for any help / direction.
If you want to objects data as JSON-documents, I recommend you to use a NoSQL database like MongoDB or CouchDB instead of MySQL, since they allow you to use JSON directly to store data.
With MySQL, the most common approach is to use a ORM layer to avoid the impedance between a relational database management system and a object-oriented language. By storing a JSON-like object in a RDMS, you are forcing your dynamic schema document to fit in a scalar value, which in my opinion is not the best solution at all.