Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 months ago.
Improve this question
I've built a simple website with authorization and registration system.
So I have user accounts and I wonder where to store profile pictures for every user.
I tried to store them in MYSQL in BLOB, but I don't think it is the best way to do this.
Most people have the opinion that files should reside on the filesystem, outside the database. But there are exceptions to every rule. There are some good reasons to keep large binaries inside the database, depending on your project requirements.
See some of my past answers on this topic:
Should I use MySQL blob field type?
What is difference between storing data in a blob, vs. storing a pointer to a file?
VARCHAR(MAX) vs TEXT vs .txt file for use in MySQL database
I also cover this in the chapter "Phantom Files" in my book, SQL Antipatterns Volume 1: Avoiding the Pitfalls of Database Programming.
The short answer is in most cases a file system (eg. the server where your code is hosted, if that's an option).
Read this short article:
Storing images in a database table is not recommended. There are too many disadvantages to this approach. Storing the image data in the table requires the database server to process and traffic huge amounts of data that could be better spent on processing it is best suited to. A file server can process such image files much better. Source: Store Images in the Database
If you do not have access to storing images on your server or enough space, you could take a look into hosting you images on an image server.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
We have log files of mySQL database and we want to use that log files for analysis (data mining, machine learning,...)
and iam very new with that.
Can you give me instruction how to do that ?
There are a number of tools that could be useful to you depending on your requirements.
But why use the MySQL logs and not the DB directly, or have the details you are searching for go to a new 'information mining' table in the DB as the user interacts with the interface. Which could be a lot more powerful
If you wish to determine the rate of inserts / queries.
You can actually set up MySQL to log these types of events to different files (so some of the work is already done, rather than having to mine a complete log file of all events).
Otherwise you are going to want to make use of tools such as Grep.
On top of that you have a system called Lucerne (from apache) that will mine the data and search for key words. They have various different hooks for going into Java, C, and others. Its very similar to how google trawl web pages.
Otherwise if you intend to mine the data 'within' the database, then the logs are clearly not your best point of call.
The logs will contain lots of information about the users also. IP address may be more tricky, but you could cross reference the name of the user running the query and the general server logs to determine the IP of the connection.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I’m writing an android app that will sync with a MySQL db on my webserver (there will also be a website reading from/writing to the same dB). The android app will store a copy of the data locally in a sqlite db to provide access while offline. If the user creates a row while offline, that record will be uploaded to the server the next time a data connection is available. I’m designing the app and website myself so I have the ability to set it up as I see fit (meaning it doesn’t have to conform to someone else’s server).
The sqlite db will have a column for id (which will represent the id as stored on the server) and a localID column. When the server receives the data, it will acknowledge the new data by returning an array (in json format) of the id numbers as stored on the server.
What would be better for this type of scenario: a transaction-safe engine or non-transaction-safe (such as isam)? It’s my understanding that isam would be faster and take less space to store but I can’t deal with losing data. I’m thinking that if the android app doesn’t receive the confirmation, it would resubmit the data. It seems like that would prevent data loss but I need a second (more-experienced) opinion. If you would go with a transaction-safe db, which would you recommend as I’ve never worked with one?
TIA!
A real database should be your default choice until you've seen that it's not fast enough.
Consider using UUIDs to generate IDs on the client that are guaranteed to be unique on the server.
Have you thought about how you would handle updates from multiple devices that both had off-line changes? You should consider some known patterns for dealing with this kind of synchronization.
Stack Overflow question
Data Replication book
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I have a question in my mind, that is it preferable to keep images in database directly using datatypes like blob,binary etc. or the way paperclip stores images by maintaining folder structure and keeping only the path in database is the standard one.
Storing pictures in the database helps you keeping your data synchronized (what if by any chance a folder name is changed manually?). Then it would save you the small effort to remember to back up data outside the database itself.
On the other side retrieving an image from a database is much slower than doing it from the file system, and database storage space I seem to remember is more expensive on a web server.
Said that, it's just a matter of choice. In case you decide to go for the database, there is a gem helping you to do that 'paperclip_database'.
There are a few advantages and disadvantages.
Advantages
Easy to protect the referential integrity. No discrepancy between image (path) saved in DB and the real image saved at the filesystem.
Database Backup also contains images. But consider you have to backup the application data nevertheless. (So this is not a good argument)
Easier rights management
Disadvantages
Performance issues. Every image has to be loaded from the database. The more images affected the more weight this argument should get.
Browser-Cache / Checking if an image has changed (If-modified-Since) not working anymore.
(But you could implement a checking yourself)
(No complete list)
My Conclution
On small sites with less traffic and images it's okay to store images in the database. On bigger sites I wouldn't do that.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I am in the middle of attempting to help my company digitize their history. One project is taking maintenance records for equipment and putting them into a mysql database. The idea is to be able to pull up a history at any time without flipping through piles and piles of paper.
My experience is limited to using phpMyAdmin to create tables and fumbling through php to output data how I want it. I've never used a relational setup.
The data fields would always be the same, the database would be populated via copy/paste from Excel (until such time comma delimited importing can be figured out), and this data would not need to be edited by endusers. It is strictly for viewing/printing purposes only.
Example fields:
id, unit number, unit_type, date, maintenance_performed
My question is, would putting all this into one table be an acceptable way to accomplish this task? Or would a relational setup be better due to the different types of units? Why?
I would focus on getting the data into the database and not on its storage. You are going to have enough problems copy-and-pasting the data in. For instance, how will you ensure that the dates are always in a consistent format?
After the data is loaded into tables, then you can worry about how to optimize it for querying purposes. How will new records continue to be uploaded? That will be a very important part of the process (I would recommend having field a creation date in the database, in addition to other information in the record).
After the data is loaded, you can worry about the best structure for organizing it. This is analogous to a real archivist, who tends to start by gathering lots and lots of data, and then figuring out the best way to organize it.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I need to create a database for dealing with click stream (from ~240 subdomains). I use a Java Script for grabbing information like (Host, Page, Date, userID, Referer, HostName, RefererPath, uniqueUserID) for each click and than insert the data to the database through a java web dynamic application. There are about 9 milion new records each day and I have to insert new records every minute. Another application needs to be able to retrieve information about pageviews/unique visitors/ect for a certain article/subdomain in the last (10min, 20min, 30min, 1hour...24 hours). I only need to keep records for the last 3 months.
Initially I thought about using MySQL as I'm only interested in open-source. But I'm thinking about NoSQL solutions. The problem is that I've had experience only with relational databases and am not really able to tell if NoSQL would be a better solution here or not. Also which database should I use if I choose to go wiht NoSQL? and would Key-value store be the best way to go?
I'm guessing this data consistency isn't critical (statistics ?) so you could indeed spare a bit of consistency. NoSQL seems a good choice and a key value store would also be my pick. Now the real question is : what is the most suited one ?
I'd give a consideration to Redis and Riak (which are basically the most well-known ones) :
Riak (AP system) :
Fault-tolerant (masterless with partitioning and replication)
Map reduce
Full text search
BASE
Redis (CP system) :
Really fast
In-memory : You need RAM ! That also means you want replication so you don't lose everything on a crash. Redis also uses disk snapshot I believe.
Master/Slave with reelection
BASE
Both have a lot more features, you should go read the documentation for gotchas. Redis is primarly used as a cache since it's fast, whereas Riak focuses on fault-tolerance. Given your scalability requirements, both can satisfy your need. Therefore you must chose according to what's above.