I have plenty of base64_encoded strings which I save in my MySQL DB. However, since base64_encoded strings are 33% larger, I would like to know how I can optimize my DB storage (Now I store my strings in a LONGTEXT field, but what about LONGBLOB or something similar?). So, how I can save my data (which is now base64 encoded) more optimized so I save storage space... (When selecting the value, I still have to be able to encode it base64 safely again).
Thanks
If you have 5.6.1 or later, FROM/TO_BASE64() are available in MySQL.
Base64 is plain ascii text. Storing base64 in a TEXT is no problem. No escaping is necessary since (I think) there are no naughty characters. But storing the result of FROM_BASE64 needs a BLOB (of some size).
So...
INSERT INTO t (myblob) VALUES ( FROM_BASE64('UmljayBKYW1lcw==') );
SELECT TO_BASE64(myblob) FROM t;
The FROM_BASE64 will reconstruct the original (pre-base64) data.
Even better would be to compress instead:
INSERT INTO t (myblob) VALUES ( COMPRESS('UmljayBKYW1lcw==') );
SELECT UNCOMPRESS(myblob) FROM t;
That way, you will (probably) get more than the 33% _if the original (pre-base64) data was compressible. If the original was a .jpg or some other already-compressed data, then you may as well use FROM_BASE64 instead of COMPRESS.
Related
I have an application where I am going to store the JSON string in MySql database through Eclipselink JPA.
The JSON string can be of any length. Most of the time a String from a JSON file of length around 200 to 300 lines.
What is the best way to store the string? To use varchar or Blob?
Please provide an example if any.
You should not save it as a BLOB, as it is primarily used for image data or other binary data... Use Varchar() or use TEXT which has a size of 65535 characters if you are unsure about how many characters you might need to store..
There was a thread previously discussing WHEN to use varchar or text: Thread
To store text - use a TEXT column (or even a LONGTEXT), blobs are for binary.
Also if you're on Mysql 5.7+ - there's now a JSON data type, which is checked for being a correct json, stored more efficiently and have pretty manipulation methods
Here's the type of text my encryption function throws out:
I generated several strings and they're never bigger than 50 characters, but I would like to give it 75 characters in mysql. I tried using varchar, but the string gets cut off because it doesn't like some characters. Any idea what data type I should use?
If it's binary data (it looks like it is), you probably should store it in a BLOB.
You can use a blob, but for short data, that will make your selects slow.
Use binary(75) or varbinary(75).
I was wondering about the best way to tackle this. I'm trying to save a user-drawn image on a HTML5 canvas to my database so I can retrieve it later.
I got as far as creating the base64 data string for the image with the following code, hooked to a simple button clickhandler:
var image_data = $("#drawing_canvas").get(0).toDataURL('image/png');
I was planning on saving that data to my database and then retrieving it later on with something like this:
var myImage = new Image();
myImage.src = imgData;
ctx.drawImage(myImage, 0, 0);
However, these base64 'strings' seem to contain a lot of data. I was wondering if there's a better way to save these images to my database? I have yet to figure out a way to save the actual image as a .png. I could get it to open as a png in a new browser tab, but that's where I'm stuck at the moment.
Or would it be fine to store these base64 data strings in my database (in, I suppose, a 'text' column)?
Thanks in advance.
You want to use a BLOB type. Here's what the MySQL docs say about it:
BLOB values are treated as binary strings (byte strings). They have no character set, and sorting and comparison are based on the numeric values of the bytes in column values. TEXT values are treated as nonbinary strings (character strings). They have a character set, and values are sorted and compared based on the collation of the character set.
http://dev.mysql.com/doc/refman/5.0/en/blob.html
I'd have some blob data such as:
0x3333332c2044e963617269652c20356520e9746167650d0a53742d4c617572656e7420285175e9626563292048344e20334d390d0a
that I'd like to convert to text because the new database has text field instead of blobs and now it makes trouble with some accentuated characters.
Is there somekind of blob to string converter somewhere?
Thanks a lot!
Try:
CONVERT(blobname USING latin1)
It depends on what the blob is. For example, I've dealt with some blobs that could be represented as basic XML files. Those would have been relatively easy to convert. However, I dealt with other blobs that were image files. If you tried to represent them as text you'd lose data.
What are in your blobs?
Create your new database with your export, once done create your text column on the table, update that using a CONVERT drop the old column, renaming the old one if required.
However if the data contains simple byte stream (that is, unstructured data, files, audio, video, whatever) and you need to represent them as pure ASCII you could change into a Base64 string.
If using phpmyadmin, tick the box that says "Dump binary columns in hexadecimal notation (for example, "abc" becomes 0x616263)" at the bottom of the export page.
I've got a database where for efficiency, i've put the data into the db in html encoded formats.
I do maintenance on the data, and then move it into production via an 'into outfile', so it ends up in a text file.
The special characters don't make it across cleanly, and it comes out as all messed up code.
Is there a way to maintain the format for the txt file?
Or should I be using another format?
The 'outfile' , and 'import'I find very efficient for doing a bulk transfer.
If i can't use that, any suggestions on the best way to find special characters in mysql?
The only thing I've found seems to find fields that ONLY contain non-ascii characters
SELECT * FROM tableName WHERE NOT columnToCheck REGEXP '[A-Za-z0-9]';
Is there a reason you're storing HTML encoded text in the database? As discussed in episode 58 of the Stack Overflow podcast, you should always try to store raw data at the highest level of precision possible.