I have table with 100000 of rows. This table also contains a BLOB field, so the table size is around 1GB. This table is scanned regularly by many queries in the application. The blob field is used only in one select query . This table also contains 5 index with size 10MB. My Doubts are.
1 ) Is it better to move the blob filed to another tables? Will this improve the speed of read operation from table?
2) The BLOB filed is used to store HTML data about 6 Kib in size. Is BLOB type apt for this?
If you can change the schema:
Store images in application server and store relative path of those images. this will result less overhead
Moving the blob field to another table can also be a good idea.
Why you are keeping html data in blob? are you seriously storing the image styles/css with this? Not recommended at all!
Related
Which one is better: having a BLOB field in the same table or having a 1-TO-1 reference to it in another table?
I'm making a MySQL database whose main table is called item(ID, Description). This table is consulted by a program I'm developing in VB.NET which offers the possibility to double-click a specific item obtained with a query. Once opened its dedicated form, I would like to show an image stored in the BLOB field, a sort of item preview. The problem is I don't know where is better to create this BLOB field.
Assuming to have a table like this: Item(ID, Description, BLOB), will the BLOB field affect the database performance on queries like:
SELECT ID, Description FROM Item;
If yes, what do you think about this solution:
Item(ID, Description)
Images(Item, File)
Where Images.Item references to Item.ID, and File is the BLOB field.
You can add the BLOB field directly to your main table, as BLOB fields are not stored in-row and require a separate look-up to retrieve its contents. Your dependent table is needless.
BUT another and preferred way is to store on your database table only a pointer (path to the file on server) to your image file. In this way you can retrive the path and access the file from your VB.NET application.
To quote the documentation about blobs:
Each BLOB or TEXT value is represented internally by a separately allocated object. This is in contrast to all other data types, for which storage is allocated once per column when the table is opened.
In simpler terms, the blob's storage isn't stored inside the table's row, only a pointer is - which is pretty similar to what you're trying to achieve with the secondary table. To make a long story short - there's no need for another table, MySQL already doesn't the same thing internally.
Most of what has been said in the other Answers is mostly correct. I'll start from scratch, adding some caveats.
The two-table, 1-1, design is usually better for MyISAM, but not for InnoDB. The rest of my Answer applies only to InnoDB.
"Off-record" storage may happen to BLOB, TEXT, and 'large' VARCHAR and VARBINARY, almost equally.
"Large" columns are usually stored "off-record", thereby providing something very similar to your 1-1 design. However, by having InnoDB do the work usually leads to better performance.
The ROW_FORMAT and the size of the column makes a difference.
A "small" BLOB may be stored on-record. Pro: no need for the extra fetch when you include the blob in the SELECT list. Con: clutter.
Some ROW_FORMATs cut off at 767 bytes.
Some ROW_FORMATs store 20 bytes on-record; this is just a 'pointer'; the entire blob is off-record.
etc, etc.
Off-record is beneficial when you need to filter out a bunch of rows, then fetch only a few. Also, when you don't need the column.
As a side note, TINYTEXT is possibly useless. There are situations where the 'equivalent' VARCHAR(255) performs better.
Storing an image in the table (on- or off-record) is arguably unwise if that image will be used in an HTML page. HTML is quite happy to request the <img src=...> from your server or even some other server. In this case, a smallish VARCHAR containing a url is the 'correct' design.
In order to speed up reading process, i thinking of transferring blob from one table to another table. But then i saw other people from stack overflow recommended to use MySQL barracuda as this method eliminate the need of transferring the blob.
My database (Design by old programmer)
Visitor_visit (table name) -> contain 5 blob column. Thought of moving the blob to another table.
So which method should i use? And how should i separate the table? Do i need to create 5 tables for each of the blob for better performance? I not really a MYSQL expect.
I wouldn't prefer to store image in the disk now. (Current situation would not let me do so)
Barracuda doesn't change how blobs are stored as much as you might think. If the blob fits on a data page with the row of other columns, it will stay together with those columns. Only if the blob is larger than what will fit on the page, is it stored off-page.
Compare with Antelope, where a large blob will also be stored mostly off-page, but the first 768 bytes of the blob will be stored on the main page with the other columns. So we're just talking about a difference in the way the blob's first 3/4 of a kilobyte is stored.
In both cases, if you simply don't reference the blob column in your query (and you shouldn't use SELECT * for this reason), InnoDB is smart enough to avoid fetching extra blob pages it doesn't need. You don't have to make another table to store blobs.
Let's see the table schema (SHOW CREATE TABLE), some clues of the sizes of the blobs, etc.
Normally, you should not bother doing anything about the 5 blobs. Everything should be automatic.
However, there are some aggregate size limitations. Again, please provide more specifics if you have hit some error message.
Meanwhile, I see no inherent reason to do "vertical partitioning".
Good day everyone, i need a suggestion how to efficient upload data from sufficiently large files into mySql database. So i have two files 5,85Gb and 6Gb of data. For uploading i have used `LOAD DATA LOCAL INTO FILE. First file still uploading (for 36hours). Current index size 7,2 GB. I have two questions :
1) The data is formatted Lile : {string , int, int, int, int}. I do not need this int values, so i created table with one field of type varchar(128), my query is LOAD DATA LOCAL INFILE "file" INTO TABLE "tale" , so will the data be correct(i mean only strings without int fields).
2) Than larger index than larger load time for next. So, an i doing something wrong? I mean, what i need. I need a fast search then in that strings(espicially in last word). So all of the strings cantains exactly 5 words does it makes any sence to put every single words in different column(n rows, 5 columns).
Please, any suggestions.
Can u drop index for now and recreate index once data is loaded on table. I think this will work.
Recreation of index will take time.
I've got a single table DB with 100K rows. There are about 30 columns and 28 of them are varchars / tiny text and one of them is an int primary key and one of them is a blob.
My question, is in terms of performance, would it be better to separate the blob from the rest of the table and store them in their own table with foreign key constraint to the primary id?
The table will eventually be turned into a sqlite persistent store for iOS core data and a lot of the searching / filtering will be done based on the NSPredicate for the lighter varchar columns.
Sorry if this is too subjective, but I'm thinking there is a recommended way.
Thanks!
If you do SELECT * FROM table (which you shouldn't if you don't need the BLOB field actually) then yes, the query will be faster because in that case pages with BLOB won't be touched.
If you do frequent SELECT f1, f2, f3 FROM table (all fields are non-BLOBs) then yes, storing BLOBS in a separate table will make the query faster because of the same reason - MySQL will have to read less pages.
If however the BLOB is selected frequently then it makes no sense to keep it separately.
This totally depends on data usage.
If you need the data everytime you query the table, there is no difference in haviong a separate table for it (as long as blob data is unique in each row - that is, "as long as the database is normalized").
If you don'T need the blob data but only metadata from other columns, there may be a speed bonus qhen querying if the blob has its own table. querying the blob data is slower thoguh, as you need to query bowth tables.
The USUAL way is not to store any blob data inside the database (at least not huge data), but store the binary data into files and have the fiel path inside the database instead. This is recommended, as binary data most likely doesn'T benefit from being inside a DBMS (not indexable, sortable, groupable, ..), so there is no drawback of storing it inside files, while the database isn't optimized for binary data ('cause, again, it can't do much with it anyway).
Blobs are stored on disk only the point to the storage is in memory in Mysql. Moving it to another table with a foreign key will not noticeably help your performance. Don't know if this is the case for sqlite.
I m designing a new forum for My Company and i wanted to confirm that saving the forum posts in MySQL Database would be scalable and would it have good Performance ..?
The Posts may have characters around 400(may be i will limit to 400 Chars). If i save 400 Chars of text in a MySQL field, and the Table has 10 million rows, will it affect performance ..?
My Main constraint is performance. Can Please Someone Shed light on this
There are two data type to consider VARCHAR or TEXT
What datatype you decide on depends on
How frequently you display it ?
Total number of characters you store
TEXT and BLOB is stored off the table with the table just having a pointer to the location of the actual storage.
VARCHAR is stored inline with the table. VARCHAR is faster when the size is reasonable, the tradeoff of which would be faster depends upon your data and your hardware, you'd want to benchmark a realworld senerio with your data.
VARCHAR (stored inline) is usually faster IF the data is frequently retrieved (included by most queries). However, for a large volume of data that is not normally retrieved (that is, not referenced by any query), then it may be better to not have the data stored inline. There is an upper limit on the row size, for data stored inline.
When a table has TEXT or BLOB columns, the table can't be stored in memory. This means every query (which doesn't hit cache) has to access the file system - which is orders of magnitude slower than the memory.
If you post content is large use TEXT field but store the Text field in a seperate table which is only accessed when you actually need it. This way the original table can be stored in memory and will be much faster.
Think of it as separating the data into one "memory table" and one "file table". The reason for doing this is to avoid accessing of the filesystem except when neccessary (i.e. only when you need the text).
You can try (posts, post_text) or (post_details, posts) or something like that.