I have a piece of information which is encoded using aes-256-cbc encryption. How should I store it in the database? Currently I'm using VARCHAR(255) utf8_bin. Is this OK or should I use other field type like VARBINARY(255)? Is there a possibility of losing some data using VARCHAR in this case? Thanks.
The possible (in)appropriateness of storing encrypted (as opposed to hashed) passwords in a database notwithstanding, AES ciphertext is binary data, and therefore should be stored as such, i.e. in a BINARY / VARBINARY column or a BLOB.
It's also possible to encode the ciphertext e.g. as base64, and then store it in a text (i.e. CHAR / VARCHAR / TEXT) column. This is less space-efficient, but it may sometimes be more convenient, e.g. when inspecting the data visually or passing it between programs that may have trouble dealing with fields containing arbitrary binary data.
I have a database column that will be storing errors/warnings returned from an API - I would usually have a column called 'description' with a varchar(255) but in some (very few cases) this column may need to store a value that is greater than 255 characters (potentially upto 400-500 characters).
This field won't need any ability to search or anything like that, can anyone suggest the best MySQL data type to use for this instance. This will store text and nothing else.
I would presume text - but just wondered what everyone else thought.
You can use varchar(500) for newer versions.
What is the MySQL VARCHAR max size?
I have a column in a table which used to be varchar(255) in the beginning and due to some design changes now it is varchar(1536) = 1024 + 512. I will not be searching or indexing this field, does it make sense to store this value in a different data type other than a varchar if you would like to optimize this for performance?
You should use TEXT like the others said, but there is some important advice every time you use TEXT or BLOB: decouple them form your base table as they really slow down accessing the table. Imagine the following structure:
CREATE TABLE article (
id INT(10) UNSIGNED,
title VARCHAR(40),
author_id INT(10) UNSIGNED,
created DATETIME,
modified DATETIME
);
CREATE TABLE article_body (
id INT(10) UNSIGNED,
body TEXT
);
Whenever you list articles you can use the article table (last 5 articles of author 33):
SELECT id, title FROM article WHERE author_id=33 ORDER BY created DESC LIMIT 5
And when someone really opens the article you can use something like:
SELECT a.title, ab.body
FROM article AS a
LEFT JOIN article_body AS ab ON ab.id = a.id
WHERE a.id=82
Yes, it will be better if you can store the values in the "TEXT" data type. For more details, please read this article.
Regarding knowledge of storage requirements, you can read this one.
Hope it helps.
You should be using a file, not a database to store this. Especially not MySQL. I made a writeup once explaining what happens if you for example download images out of a database BLOB, see http://mysqldump.azundris.com/archives/36-Serving-Images-From-A-Database.html. Using files, you can use the web server fast path using the sendfile(2) system call, and it is much faster to use this.
MySQL also has no BLOB API. That means, it is impossible to upload or download objects larger than max_allowed_packet, and it is hard to work your way around that using SUBSTRING(), because that will make needless copies of strings in server memory.
If you absolutely MUST store BLOB or TEXT data in the server, you have the choice of TINYTEXT, TEXT, MEDIUMTEXT and LARGETEXT which are limited to 255, 65535, 16 MB and 4GB of data in the server, additionally constrained by max_allowed_packet.
Large BLOB or TEXT information will completely wreck data density in your table. It is useful to create an artificial 1:1 or 1:0 relationship to a BLOB table, and then store the blobs in this extra table.
When MySQL shows a query plan that is 'using tempoary', it means that the server needs to materialize the result set table in the server before delivering the result. This is being done using MEMORY tables, if possible. Any TEXT or BLOB type cannot be represented in MEMORY tables, hence the temporary table then hits the disk as a MyISAM table instead.
You need to scan for such query plans, and convert them into something that loads the ID values of the BLOB/TEXT values instead. In a second query, you'd then SELECT id, thetext FROM texttable WHERE id in ( ... ) to get the TEXT/BLOB values. That will make the query with 'using temporary' not use TEXT or BLOB types, and you can get the TEXT fields then with a trivial query that runs without 'using temporary'.
You can learn more about the internals of MySQL TEXT and BLOB storage by reading http://www.mysqlperformanceblog.com/2010/02/09/blob-storage-in-innodb/
I am here after get stuck in my development.
In my website:
I used TEXT datatype in MYSQL to store the content of the textarea given by user input. Today I give a large text (112KB) in the textarea to store in DB.
and it breaks.
Then I read from internet -
TINYTEXT: 255 characters - 255 B
TEXT: 65,535 characters - 64 KB
MEDIUMTEXT: 16,777,215 - 16 MB
LONGTEXT: 4,294,967,295 characters - 4 GB
For me I have to switch to the MEDIUMTEXT.
I would use text for columns with variable length.
I am creating a statistics module in SQL Server 2008 that allows users to save data in any number of formats (date, int, decimal, percent, etc...). Currently I am using a single table to store these values as type varchar, with an extra field to denote the datatype that it should be.
When I display the value, I use that datatype field to format it. I use sprocs to calculate the data for reporting; and the datatype field to convert to the appropriate datatype for the appropriate calculations.
This approach works, but I don't like storing all kinds of data in a varchar field. The only alternative that I can see is to have separate tables for each datatype I want to store, and save the record information to the appropriate table based on datatype. To retreive, I run a case statement to join the appropriate table and get the data. This seems to solve. This however, seems like a lot of work for ... what gain?
Wondering if I'm missing something here. Is there a better way to do this?
Thanks in advance!
Assumedly, when you pull this data from the database, it goes through some kind of normalisation before being displayed to make the report useful? If so, can it not be normalised before it goes to the database?
I have a project where I need to store a large number of values.
The data is a dataset holding 1024 2Byte Unsigned integer values. Now I store one value at one row together with a timestamp and a unik ID.
This data is continously stored based on a time trigger.
What I would like to do, is store all 1024 values in one field. So would it be possible to do some routine that stores all the 1024 2byte integer values in one field as binary. Maybe a blobfield.
Thanks.
Br.
Enghoej
Yes. You can serialize your data into a byte array, and store it in a BLOB. 2048 bytes will be supported in a BLOB in most databases.
One big question to ask yourself is "how will I need to retrieve this data?" Any reports or queries such as "what IDs have value X set to Y" will have to load all rows from the table and parse the data AFAIK. For instance, if this were user configuration data, you might need to know which users had a particular setting set incorrectly.
In SQL Server, I'd suggest considering using an XML data type and storing a known schema, since this can be queried with XPath. MySQL did not support this as of 2007, so that may not be an option for you.
I would definitely consider breaking out any data that you might possibly need to query in such a manner into separate columns.
Note also that you will be unable to interpret BLOB data without a client application.
You always want to consider reporting. Databases often end up with multiple clients over the years.