I am working on an application that plays audio and video clips. I've read other posts that recommend storing the index to the video file in the database and not the video itself. However, right now the biggest video I have is around 1MB and I have around 10 videos right now. Would I be better off just by storing the actual videos in the database. I also have text instructions that appear on the screen. Should I store the text directly in the database or have indexes pointing to their files?
It is entirely up to you, however, the advantage of not storing the binary (file) in the database is that you can use other servers to serve the video. This often results in faster loading times (file servers are optimized to serve media, see http://cloudflare.com ), reduced load on your database (even though it's just 1MB), etc.
Another point, is that if you're storing the binary on your database, when you're sending the data/video to the client/user you'll usually have to set the headers and mime types so that the client knows how to display the video. It's not difficult to do, but it's just extra work (maintenance, and possible trouble-shooting areas in the future) that's already done for you that you don't need to do/worry about when you use a file/media server.
The database is best for information query, so I think it's best to store the text and indices on there and put your video elsewhere. It may help you when you get a bit more complicated or have more videos to manage.
Related
I plan to make rtsp server that provides video or audio, saved live, such as camera video data.
So I'm searching how to save these data.
It will be quite a lot of files, about 10 ~ 10000. Each of that file size will be around 4GB.
First time, I think each of files made just files and write index data to DB.
It is the best simple and easy work i think.
But how about save data directly to DB??? such as using blob.
I seemed it looks much more easier than first thing if i deal DB well.
Of course I searched in case of MySQL, MSSQL.
Most of them are negative. then how about big database? such as hadoop or NoSQL?
i decided only index in db!
there is so much disadvantages insert video data in db.
My proposed solution.
1) You will store videos in content delivery network (CDN). Have a location of video.
2) In your NoSQL database like HBase, store ImageId (or Image Name) and Location of Image.
3) Other solution : Use Solr Indexing coupled with Cassandra. Images are stored in Cassandra & Index is stored in Solr. You will get images from Cassandra using Solr index.
This question already has answers here:
Storing Images in DB - Yea or Nay?
(56 answers)
Closed 8 years ago.
I'd like to know what's the best method to store images into a MySQL database with the best performance. (assume a big database)
Solution A: Use LONGBLOB type and put the binary content
Solution B: Store the files on the hosting and save the URL how a VARCHAR type
Thanks in advance
If you are building a system that needs to scale up to serve many images: there is an enormous disadvantage to putting the image contents in your DBMS.
Web servers can be clustered around large common file storage systems and can serve images very efficiently. This sort of file-serving architecture has been highly optimized by the various web server products like Apache, nginx and IIS.
But if you put your images in database BLOBs, fetching them from the database becomes a bottleneck. Plus you'll need a script (php, .net, whatever) to run to serve each image.
Almost all production web sites serve their images directly from files. You can store, in your DBMS, the locations on your file store, then convert them to URLs in the HTML pages you send out.
For example, if the imgloc column contains u/10234/abcde.img, and the table also has width and height columns, your web app can emit something like
<img src="/content/u/10234/abcde.img" width="300" height="200">
Then the client will fetch the image from your content store.
I'd say rather than store the url, store the image name like "image1.png" and host the images on your server. That should help reduce db size and call times a bit. Then programmaticly prefix the url path and reference that.
Its probably also better to just save the file name in case the image location changes. that way you won't have to worry about updating the entire table and you can just fix the one line of code with the url path
I tend to lean toward putting them in the database. That way when you do backups of the DB, it's all packaged up together. If you ever move servers, then you don't have to worry about moving files from the file system.
Unless you're talking about large files(GBs), i doubt you would see much of a difference from file system vs.db performance.
Either way is going to work for you, just a matter of preference and what works best for your particular situation.
There are many answers already on SO and the internet.
Storing Images in DB - Yea or Nay?
I'm looking to store a video in a database as a BLOB type.
The video will be playing back using progressive downloading.
Are there any problems that I might face?
Using the html5 video player it would look like this:
<video width="800" height="450" src=BLOB type=""/></video>
By storing video files in the database it means your data-tier is invoked every time there's a request for the video which will be very expensive from an IO perspective, also given that (AFAIK) most database libraries don't let you access binary information from an offset it means you won't be able to provide seek support to consumers (they'd all have to download the video fully first).
Storing video-files in a database can be acceptable depending on the load (how many requests will be made for this video?), if you're employing any form of caching so it doesn't have to hit the database so often (e.g. storing the whole video in memcached), how big the videos are (if they're just 6-second Vine-style videos then you'll get away with it, if these are 20GB 4K hi-def documentary videos then probably not).
Out of curiosity, why not use a third-party video hosting service like YouTube or use a CDN to host your video files?
Some downside of storing Blobs
When your DB grows bigger and bigger it will become harder to backup. Restoring a backup of a table with over 100 GB of data is not something that makes you happy.
Another thing that get is that all the table management functions get slower and slower as the dataset grows.
But this can be overcome by making your data table just contain 2 fields: ID and BLOB.
Retrieving data (by primary key) will likely only become a problem long after you hit a wall with backing up the dataset.
We had similar situation in pas project where we saved video and audio on disk and researched if there are any benefits to keep them in database. What we've found is that file system is easiest and primitive solution, whenever you have complex requirement your choice is database, but keep in mind these:
Saving on disk offers simplicity, i.e. you can later change the structure of folders and move files between them relatively easy. Typically you would want to distinguish between audio and video files, corrupted files, formats of files, group files by date, etc. and keep those in sub-folders. So whenever you need more folders you can easily add them
Think in advance about retain policies when for example, newest media you would keep for 30 days, then move into different disk or whatever. You need clear understanding how you would achieve this with database
CDN. It was really easy to spread out files through servers. With database it depends on your scenario and might be easier or more challenging
Database will add overhead as you can not just access the file through URI and need some middle-ware code that will communicate to db
Database will be quite helpful if you need analysis of data or Business Intelligence support, transactions and indexing
I have a website in which I let users upload images. I convert these images to data URLs through HTML5 and store them as text fields in a database: http://en.wikipedia.org/wiki/Data_URI_scheme
I figured this would reduce the time for page load since I need to make fewer HTTP requests even though the main HTML page would be much longer.
I'm suspicious though that these images stored in URLs take up more space in the database than their static counterparts on disk. I noticed that a data URL for image had 250K characters (so I assume stored in 250KB), but when I right-clicked and saved the same image on disk, the image was only 180K.
Do data URLs significantly inflate the memory required to store an image?
Yes. Images stored as Data URIs go through a base64_encode process. Which inflates their size by approximately 30%.
You're approaching this wrong.
Store the files themselves on the disk, separated by folders. I like naming the images as hashes of their contents. That way you can avoid duplicates easily if you want.
Store data on those images on the database. Include the path, upload time, uploading user, position of Saturn's moons, and whatever other data you may want on the database, except for the image itself, which should be stored on the file system.
If you wish, you can generate the Data URI in real time (like I said, it's a simple base64_encode).
It can also take more than 250KB of space if you're using a multi-byte character encoding for your DB column. If they are 250K characters, it may be using between 500K and 1M of disk space.
Not only do they take up more space, but they can not be cached by the users browser independently of the page they are on. This can reduce performance significantly.
it's definitly not worth it memorywise. as said in other answers:
base64 encoded grow by about 30% (Second Rikudos Answer)
I would not recommend storing image data in a database at all, but there is controversy, see here
serving base64 encoded dataURLs in your main html may be ok for small images, but as we talk about user uploaded images, I doubt they are lightweight (no more than 10k, depends a bit). also, Daniels note about caching is weighting strong. but in fact, that's another question already.
I have been contracted to store user-contributed photos for a contest run through Facebook. I'm currently having issues uploading files from Facebook to my server (I understand they strip out file variables in post requests, but all the answers I have seen simply say "use an iframe". My app is set up to be an iframe (vs. FBML, in application settings).
This is proving to be very time consuming to troubleshoot and get up and running (I tried selling just URL storage of remote pictures, which was turned down). I am considering just saving the images as a blob in my mysql database. We expect to get around ~2,000 photos of various size. What's the general take on this type of load? I've read various SO threads discussing multiple TBs of data, and filesystem being a better choice, but for maybe a gig or two, is this unreasonable?
Thanks
Given the number of photos it doesn't sound unreasonable to store them as blobs. I have shoe-horned similar numbers of megabyte sized pics into SQL Server with no detrimental effects to my system. However YMMV and I'd suggest that you code up a simple MySQL Db and shove 2 or 3,000 pics of the applicable size in and see how your system behaves. You should be able to do that in a very short amount of time.