I found information about this already, but of more general kind and focused on "if the data shuld change a lot...". I will try to be one step more specific here.
I am developing a web application. It's possible to configure what should be presented or not. E.g. In a form, there can be a number of different drop-down lists, but it should be configured which drop-down lists should be presented.
Hence, it's going to be a lot of reading of the config info. Updating the configuration will be done very seldom. Also, the configuration itself should be performed with using a web application as well.
What's the best strategy, using files or database for the config data?
I guess this depends on if you are already using a database for the rest of the web application. If you are then it makes sense to just add another table. Otherwise the overhead of setting up a database server and managing connections just for configuration is too much. In which case a flat file using structured text is probably your best bet.
If you are already using a database, you could cache the results so that the overhead of looking up the results is lower, then clear the cache when the config is updated.
The best strategy is encapsulation.
If you encapsulate access to your configuration data properly, you'll be able to start off with whichever implementation meets your short term requirements, safe in the knowledge that you can change it later.
Up until I read the requirement of
the configuration itself should be performed with using a web application,
I'd have said a flat file or PHP include would have sufficed, but given that requirement (and the availability of MySQL), I'd say use a database.
Plus, you never know when the config's update frequency will increase.
Related
For my simple App, i have a ftp server where i can store file (json or xml) or DB. Multiple clients could access that file or DB to read or write (DB or file would have only up to 100 entries).
From one point of view, DB is more suited for having big amounts of entries, due to indexing. But from other point of view, i am not sure if there would be issue with xml or json file if multiple clients try to read or write at the same time from the same file? So i am thinking to use DB just to avoid that issue.
I'd suggest using a database for a few reasons:
Databases are designed for exactly this scenario.
If you ever need to work on a larger scale you won't need to change your code.
You'll get to practice writing db code that will be useful in future, larger scale projects.
There are some really good database technologies that will work well with what you need, for example MongoDB, MySQL, SQL Server. They all have great support, code examples and you'll be able to use Stack Overflow to ask questions about them.
After googling, it seems that SQLite is the best choice. It is good approach for small DB, it is self-contained, allowing safe access from multiple processes or threads. Exactly what i needed.
I am making a new version of a old static website that grew up to a 50+ static pages.
So I made a JSON file with the old content so the new website can be more CMS (with templates for common pages) and so backend gets more DRY.
I wonder if I can serve that content to my views from the JSON or if I should have it in a MySQL database?
I am using Node.js, and in Node I can store that JSON file in memory so no file reading is done when user asks for data.
Is there a correct practise for this? are there performance differences between them serving a cached JSON file or via MySQL?
The file in question is about 400Kb. If the filesize is relevant to the choice of one tecnhology over the other?
Why add another layer of indirection? Just serve the views straight from JSON.
Normally, database is used for serving dynamic content that changes frequently, records have one-to-many or many-to-many relationships, and you need to query the data based on various criteria.
In the case you described, it looks like you will be OK with JSON file cached in server memory. Just make sure you update the cache whenever content of the file changes, i.e. by restarting the server, triggering cache update via http request or monitoring the file at the file system level.
Aside from that, you should consider caching of static files on the server and on the browser for better performance
Cache and Gzip static files (html,js,css,jpg) in server memory on startup. This can be easily done using npm package like connect-static
Use browser cache of the client by setting proper response headers. One way to do it is adding maxAge header on the Express route definition, i.e:
app.use "/bower", express.static("bower-components", {maxAge:
31536000})
Here is a good article about browser caching
If you are already storing your views as JSON and using Node, it may be worth considering using a MEAN stack (MongoDB, Express, Angular, Node):
http://meanjs.org/
http://mean.io/
This way you can code the whole thing in JS, including the document store in the MongoDB. I should point out I haven't used MEAN myself.
MySQL can store and serve JSON no problem, but as it doesn't parse it, it's very inflexible unless you split it out into components and indexing within the document is close to impossible.
Whether you 'should' do this depends entirely on your individual project and whether it is/how it is likely to evolve.
As you are implementing a new version (with CMS) of the website it would suggest that it is live and subject to growth or change and perhaps storing JSON in MySQL is storing up problems for the future. If it really is just one file, pulling from the file system and caching it in RAM is probably easier for now.
I have stored JSON in MySQL for our projects before, and in all but a few niche cases ended up splitting up the component data.
400KB is tiny. All the data will live in RAM, so I/O won't be an issue.
Dynamically building pages -- All the heavy hitters do that, if for no other reason than inserting ads. (I used to work in the bowels of such a company. There were million of pages live all the time; only a few were "static".)
Which CMS -- too many to choose from. Pick a couple that sound easy; then see if you can get comfortable with them. Then pick between them.
Linux/Windows; Apache/Tomcat/nginx; PHP/Perl/Java/VB. Again, your comfort level is an important criteria in this tiny web site; any of them can do the task.
Where might it go wrong? I'm sure you have hit web pages that are miserably slow to render. So, it is obviously possible to go the wrong direction. You are already switching gears; be prepared to switch gears a year or two from now if your decision turns out to be less than perfect.
Do avoid any CMS that is too heavy into EAV (key-value) schemas. They might work ok for 400KB of data, but they are ugly to scale.
Its a good practice to serve the json directly from the RAM itself if your data size will not grow in future. but if data is going to be increased in future then it will become a worst application case.
If you are not expecting to add (m)any new pages, I'd go for the simplest solution: read the JSON once into memory, then serve from memory. 400KB is very little memory.
No need to involve a database. Sure, you can do it, but it's overkill here.
I would recomend to generate static html content at build time(use grunt or ..). If you would like to apply the changes, trigger a build and generate static content and deploy it.
I'm working on a Wordpress theme and it needs to have some settings. Until now I have them in the database but I also want them to be portable. For that reason I also saved the data as an array into a settings.php-file. Now I'm considering to don't use the database at all to avoid storing things twice.
Some questions about this
Are there any bad thing about storing data in an array within a included PHP-file, compared to MySQL database? (It's just settings, nothing needs to be sorted, no relations needed)
Which is fastest? Include a php-file with an array, or load data from the database?
Other thougts about this?
I give a check-vote to the most complete answer to my questions. Short answers might get a vote up.
I'm a pretty hardcore database guy and I would say they do not need to be in a database.
The clues are in your statements "It's just settings, nothing needs to be sorted, no relations needed" and "I also want them to be portable"
My main argument is simplicity. PHP is extremely good with arrays, it likes them, it understands them, can easily load them from files and save them to files. So even if you do change them from time to time from the app, updating and saving an array is no big deal. So, if you use the array, you use a native feature of PHP and that creates architectural simplicity for this feature.
So for portability, the most portable database is the one you do not use. When you have the simplicity of using a native PHP data format, you don't need the database (at least not for this)
For speed, on Linux anyway PHP can open a file and read it faster than it can make a roundtrip to the database for anything.
The only remaining argument against an array solution would be interaction with other data, but you have said there is none.
So, as a hardcore database guy, I would say do not use the database just because it is there. Databases are incredible for structured data, if this is just a flat list of settings it is not structured data. The db can do, the filesystem can do it, pick what is simpler.
Don't create a file based storage for your Wordpress-Theme-Settings if you want it to be portable. Some sites might have the themes folder readonly.
For the initial setup it's ok to read your settings from almost everything filebased. Later on its best stored (and backuped) in the database.
If you store things in a file use the ini-style based files as PHP gives you an API for free. Things usually only tend to be better stored in an array or serialized if your options are not restricted.
Don't care about performance too much, simply use the wordpress options api as a best practice.
If you're worried about portability, you might be interested in using ODBC or PHP Data Objects
As for which is fastest, I'm no expert, but the settings file only involves reading a text file and parsing it. The database option usually will result in TCP connections (unless you use mysqlite, which I would recommend if you are going to store more than just file paths and database names.
Regards,
The main problem of having them in a file occurs if you want the settings to be configurable via the website itself. If not, then having them in the code is no real cause for concern.
I have a client software program used to launch alarms through a central server. At first it stored configuration data in registry entries, now in a configuration XML file. This configuration information consists of Alarm number, alarm group, hotkey combinations, and such.
This client connects to a server using a TCP socket, which it uses to communicate this configuration to the server. In the next generation of this program, I'm considering moving all configuration information to the server, which stores all of its information in a SQL database.
I envision using some form of web interface to communicate with the server and setup the clients, rather than the current method, which is to either configure the client software on the machine through a control panel, or on install to ether push out an xml file, or pass command line parameters to the MSI. I'm thinking now the only information I would want to specify on install would be the path to the server. Each workstation would be identified by computer name, and configured through the server.
Are there any problems or potential drawbacks of this approach? The main goal is to centralize configuration and make it easier to make changes later, because our software is usually managed by one or two people at most.
Other than allowing for the client to function offline (if such a possibility makes sense for your application), there doesn't appear to be any drawback of moving the configuration to a centralized location. Indeed even with a centralized location, a feature can be added in the client to cache the last known configuration, for use when the client is offline).
In case you implement a [centralized] database design, I suggest to consider storing configuration parameters in an Entity-Attribute-Value (EAV) structure as this schema is particularly well suited for parameters. In particular it allows easy addition and removal of particular parameters and also the handling parameters as a list (paving the way for a list-oriented display as well in the UI, and therefore no changes needed in the UI either when new types of parameters are introduced).
Another reason why configuartion parameter collections and EAV schemas work well together is that even with very many users and configuration points, the configuration data remains small enough that is doesn't suffer some of the limitations of EAV with "big" tables.
Only thing that comes to mind is security of the information. In either case you probably have that issue though. Probably be easier to interface with though with a database as everything would be in one spot.
I'm not really asking whether I should use either a RDBMS or config files for 100% of my application configuration, but rather what kind of configuration is best addressed by each method.
For example, I've heard that "any kind of configuration that is not changeable by the end-user" should be in config files rather than the database. Is this accurate? How do you address configuration?
(I'm primarily concerned with many-user web applications here, but no particular platform.)
I find that during development it is of great benefit to have configuration stored in a file.
It is far easier to check out a file (web.config, app.config, or some custom file) and make changes that are instantly picked up when the code is run. There is a little more friction involved in working with configuration stored in a database. If your team uses a single development database you could easily impact other team members with your change, and if you have individual databases it takes more than a "get latest" to be up and running with the latest configuration. Also, the flexibility of XML makes it more natural to store configuration that is more than just "name-value" pairs in a file than in a relational DB.
The drawback is where you want to reuse the configuration across multiple apps or web site instances. In my own case, we have a single config file in a well-known location that can be referenced by any application.
At least, this is how we store "static" configuration that does not have to be updated by the system at runtime. User settings are probably more suited to storage in the DB.
The oneliner: As a general principle - the more likely the config data should change the better to put it into db.
The legal disclaimer:
You would need to have almost always a kind of "bootstrapping" configuration, which must be saved into a file, thus if you are using a db to store your configuration the size of the "bootrapping" conf would depend on the other great principle:
"Work smarter not harder !!!"
One thing to conside is how much config data there is, and perhaps how often it is likely to change. If the amount of data is small, then saving this in a database (if your not already using a db for anything else), would be overkill, equally maintaining a db for something that gets changed once every 6 months would probably be a waste of resources.
That said, if your already using a database for other parts of your site, then adding a table or two for configuration data is probabley not a big issue, and may fit in well with the way you are storing the rest of your data. If you already have a class for saving your data to a db, why write a new one to save to a config file.