I'll use the read operation many times.
According to some tests I did, MySQL is fatest in reading one single row than MongoDB. Is it better to use MySQL?
also, i heard someong saying that sometimes is better storing session in files, is it true?
If you tested it and it was faster then why are you not trusting your own research? You can save the data in a relational database like MySQL, a JSON document store like MongoDB, or a flat file (the flat file may run into i/o contention to open &n read the file). Go with what you are comfortable with and trust your research.
Related
I'm currently using a MySQL where do I store JSON information and I'd fetch them from MySQL and parse them on my application. I would like to get rid of MySQL, but first I would like to know is that wise?
Is that efficient if I move to way that I store the data into data folder that contains .json files and these contains the data I need? There will be my app's coordinate data per user who wants to track themselves on map. Will that cause any issues? I don't need "query", but what about big data like 50K lines in example? Same amount will be in MySQL too. Amount doesn't change, but will there be any problems that appears when moving from "reading from sql to reading from json files"
It's difficult to answer all these questions in one, but I'll address some of them:
There are dedicated NoSQL databases that are very good at the type of data storage you're talking about: MongoDB, CouchDB etc. It might be worth checking these out. They are very good at dealing with JSON data. Querying and parsing are very simple in Node.js.
You can store JSON in MySQL (or other RDMS systems), I've done it in several projects with good results. As of MySQL 5.7.8 there is a dedicated JSON type. Queries can actually work surprisingly well, I know I've queried tables with tens of millions of JSON entries pretty quickly.
Make sure you consider backup and restore scenarios, what happens in the event of a data loss. Using MySQL or a NoSQL database will simplify this for you. Either way make sure you have this covered!
I wouldn't call 50K lines big data! I dealt with databases with tens of millions of rows.. this still wouldn't be called big data.
I would probably not recommend storing your data in files. I've worked in telematics before, we stored millions of JSON blobs in relational databases with very little problems. Later on we planned to move to a NoSQL database for these, but the relational database worked surprising well, especially because you can adopt a hybrid approach of using relational queries and including JSON data in the results (to be parsed by clients).
You might not need the ability to query, but it's very useful to get for example "Give me all JSON for user id 100". An RDBMS or NoSQL system would make this very easy.
I'm being told this question is subjective, but hey ho.
Am I best storing user activity in a table in a mysql database or in an xml file. The aim is for the data to be printed on their account page.
I'm worried that I will either end up with a huge/slow database or many many xml files on the server (one for each user).
Thanks
Use a DB of some sort. Files may have issues regarding I/O, locking, concurrent access and so on.
If you do use files, prefer json over xml.
For RDMS, Mysql is fine.
I would suggest using a NOSQL, my choice would be Redis.
Store it in a table. If you're storing billions of records you'll want to investigate partitioning or sharding, but those are problems you should tackle if and only if you will be hitting limits.
Test any design you have by simulating enough user activity to represent a year or two worth of vigorous use. If it holds up, you're okay. If not you'll have specific problems to address.
Remember in tables of this sort having indexes is important for retrieval speed, but too many indexes can slow down inserts. There's a balance here between too much and too little indexing you'll have to find.
XML files are often extremely expensive to append to unless you do something like what Adium did with their reverse XML parser built to append to XML logs efficiently.
I suggest it should be on the DB.
1) As it would be much easier to maintain a Database table for log information than separate log files. not much load on the server.
2) for RDBMS you need to query for those user log history which would be hard for the xml files
3) Proper indexing will help for faster data retrieval.
4) XML read/write cost more I/O OP
I read some where that it is better to use Redis as cache server,because Redis holds the data in memory,so if you are going to save lots of data,Redis is not a good choice. Redis is good for keeping temporary data.now my question is:
1.where do rest of databases (especially neo4j and sql server) save data?
Don't they save data in memory?
if no,so where they save them?
if yes,why do we use them for saving lots of data?
2."It is better to save indices/relationships in neo4j and data in mysql,and retrieve the index from neo4j and then take the data related to the index from mysql" (I have read it some where),is this because neo4j has the same problem as Redis does?
Neo4J and SQL Server both store data on the file system. However, both also implement caching strategies. I am not an expert on the caching in these databases. Usually you can expect recently accessed data to be cached and data that has not been accessed for a while to fall out of the cache. If the DB needs to get something that is in the cache, it can avoid hits to the file system. Neo4j saves data in a subfolder called "data" by default. This linke may help you find the location of a SQL Server database: http://technet.microsoft.com/en-us/library/dd206993.aspx
This will depend a lot on your specific use-case and the required performance characteristics. My gut feeling is to put data in one or the other based on some initial performance tests. Split the data up if it solves some specific problem.
I'm working on a Wordpress theme and it needs to have some settings. Until now I have them in the database but I also want them to be portable. For that reason I also saved the data as an array into a settings.php-file. Now I'm considering to don't use the database at all to avoid storing things twice.
Some questions about this
Are there any bad thing about storing data in an array within a included PHP-file, compared to MySQL database? (It's just settings, nothing needs to be sorted, no relations needed)
Which is fastest? Include a php-file with an array, or load data from the database?
Other thougts about this?
I give a check-vote to the most complete answer to my questions. Short answers might get a vote up.
I'm a pretty hardcore database guy and I would say they do not need to be in a database.
The clues are in your statements "It's just settings, nothing needs to be sorted, no relations needed" and "I also want them to be portable"
My main argument is simplicity. PHP is extremely good with arrays, it likes them, it understands them, can easily load them from files and save them to files. So even if you do change them from time to time from the app, updating and saving an array is no big deal. So, if you use the array, you use a native feature of PHP and that creates architectural simplicity for this feature.
So for portability, the most portable database is the one you do not use. When you have the simplicity of using a native PHP data format, you don't need the database (at least not for this)
For speed, on Linux anyway PHP can open a file and read it faster than it can make a roundtrip to the database for anything.
The only remaining argument against an array solution would be interaction with other data, but you have said there is none.
So, as a hardcore database guy, I would say do not use the database just because it is there. Databases are incredible for structured data, if this is just a flat list of settings it is not structured data. The db can do, the filesystem can do it, pick what is simpler.
Don't create a file based storage for your Wordpress-Theme-Settings if you want it to be portable. Some sites might have the themes folder readonly.
For the initial setup it's ok to read your settings from almost everything filebased. Later on its best stored (and backuped) in the database.
If you store things in a file use the ini-style based files as PHP gives you an API for free. Things usually only tend to be better stored in an array or serialized if your options are not restricted.
Don't care about performance too much, simply use the wordpress options api as a best practice.
If you're worried about portability, you might be interested in using ODBC or PHP Data Objects
As for which is fastest, I'm no expert, but the settings file only involves reading a text file and parsing it. The database option usually will result in TCP connections (unless you use mysqlite, which I would recommend if you are going to store more than just file paths and database names.
Regards,
The main problem of having them in a file occurs if you want the settings to be configurable via the website itself. If not, then having them in the code is no real cause for concern.
Most projects now need some form of a database. When someone says database, I usually think relational databases, but I still hear about flat file XML databases.
What parameters do you take into consideration when deciding between a "real" database and a flat-file XML database. When should one be used over the other, and under what circumstances should I never consider using a flat file (or vice versa a relational) database?
There is no such thing as a xml flat file database. Flat xml files are non-databases in that they have no higher functions like indices - have fun with larger datasets and searches or analytical queries without any index.
XML databases are another topic and may have their needs (content management, document storage in general - complicated schemata you dont care too much from the database point of view).
Flat files are fine for things like settings 8smaller files), but a real database is a real database. ACID conditions are hard to guarantee for flat files.
To add to Rachel's answer.
concurrency
read vs. write
If you have something simple that's going to be read often and is not going to change much it might be more optimal to use a flat file and save the overhead.
On the other hand, if you have to support multiple connections that are going to be adding and updating the data you'll want to use a database.
Few Parameters to consider is
Amount of Data
Complexity of Data
Relationship between Data
If we have less amount of Data with low Complexity and no inter-relationship than people would go for Flat file but in real application this is rarely the case and so you will always find Relational Databases used very often.
XML file is not a database. Read Joel's "Back to basics" article to see the difference.