I have a 2 tables, one in which I have groups, the other where I set user restrictions of which groups are seen.
When I do LEFT JOIN and specify no condition, it shows me all records. When I do WHERE group_hide.hide!='true' it only shows these records that have false enum type set to them. With JOIN, other groups get the hide field set as "NULL".
How can I make it so that it excludes only these that are set to true, and show everything else that has either NULL or false?
In MySQL you must use IS NULL or IS NOT NULL when dealing with nullable values.
HEre you should use (group_hide.hide IS NULL OR group_hide.hide != 'true')
Don already provided good answer to the question that you asked and will solve your immediate problem.
However, let me address the point of wrong data type domain. Normally you would make hide be BOOLEAN but mysql does not really implement it completely. It converts it to TINYINT(1) which allows values from -128 to 127 (see overview of data types for mysql). Since mysql does not support CHECK constraint you are left with options to either use a trigger or foreign reference to properly enforce the domain.
Here are the problems with wrong data domain (your case), in order of importance:
The disadvantages of allowing NULL for a field that can be only 1 or 0 are that you have to employ 3 value logic (true, false, null), which btw is not perfectly implemented in SQL. This makes certain query more complex and slower then they need to be. If you can make a column NOT NULL, do.
The disadvantages of using VARCHAR for a field that can be only 1 or 0 are the speed of the query, due to the extra I/O and bigger storage needs (slows down reads, writes, makes indexes bigger if a field is part of the index and influences the size of backups; keep in mind that none of these effects might be noticeable with wrong domain of a single field for a smaller size tables, but if data types are consistently set too big or if the table has serious number of records the effects will bite). Also, you will always need to convert the VARCHAR to a 1 or 0 to use natural mysql boolean operators increasing complexity of queries.
The disadvantage of mysql using TINYINT(1) for BOOL is that certain values are allowed by RDBMS that should not be allowed, theoretically allowing for meaningless values to be stored in the system. In this case your application layer must guarantee the data integrity and it is always better if RDBMS guarantees integrity as it would protect you from certain bugs in application layer and also mistakes that might be done by database administrator.
an obvious answer would be:
WHERE (group_hide.hide is null or group_hide.hide ='false')
I'm not sure off the top of my head what the null behaviour rules are.
Related
I have a table structure shown below contains Structure of Roles Table I taken:
Let it be a "roles" table contains some records related to roles of users.
Now here I have taken one column "is_archived(int)" which I am using to get to know that role still exists or deleted.
So I am considering two values for that column:
"NULL"=> if that role still exists (like TRUE),
"1" => if deleted /inactive (like FALSE)
For my table maximum records will contain "NULL" value for this column and Default value is also "NULL".
Now I am in a dilemma that is there any performance issue in this case as I am using "NULL" instead of "0".
I need to know the pros and cons of this case(Like "Search Performance", "Storage", "indexing", etc).
And in case of cons, what are the best alternatives?
My opinion is that NULL is for "out of band", not for kludging an in-band value. If there is any performance or space difference, it is insignificant.
For true/false, use TINYINT NOT NULL. It is only 1 byte. You could use ENUM('false', 'true'); it is also 1 byte.
INT, regardless of the number after it, takes 4 bytes. Don't use INT for something of such low cardinality.
Leave NULL to mean "not yet known" or any other situation where you can't yet say "true" or "false". (Since you probably always know if it is 'archived', NULL has no place here.
You could even use ENUM('male', 'female', 'decline_to_state', 'transgender', 'gay', 'lesbian', 'identifies_as_male', 'North_Carolina_resident', 'other'). (Caveat: That is only a partial list; it may be better to set up a table and JOIN to it.)
I agree with #RickJames about NULL. Don't use NULL where you mean to use a real value like true. Likewise, don't use a real value like 0 or '' to signify absence of a value.
As for performance impact, you should know that to search for the presence/absence of NULL you would use the predicate is_archive IS [NOT] NULL.
If you use EXPLAIN on the query, you'll see that that predicate counts as a "range" access type. Whereas searching for a single specific value, e.g. is_archive = 1 or is_archive = 0 is a "ref" access type.
That will have performance implications for some queries. For example if you have an index on (is_archived, created_on) and you try to do a query like:
SELECT ... FROM roles
WHERE is_archived IS NULL AND created_on = '2017-01-31'
Then the index will only be half-useful. The WHERE clause cannot search the second column in the index.
But if you use real values, then the query like:
SELECT ... FROM roles
WHERE is_archived = 0 AND created_on = '2017-01-31'
Will use both columns in the index.
Re your comment about NULL storage:
Yes, in the InnoDB storage engine, internally each row stores a bitfield with 1 bit per column, where the bits indicate whether each column is NULL or not. These bits are stored compactly, i.e. one byte contains up to 8 bits. Following the bitfield is the series of column values. A column that is NULL stores no value. So yes, technically it is true that using a NULL reduces storage.
However, I urge you to simplify your data management and use false when you mean false. Do not use NULL for one of your values. I suppose there's an exception if you manage data at a scale where saving one byte per row matters. For example, if you are managing tens of billions of rows.
But at a smaller scale than that, the potential space savings aren't worth the extra complexity you add to your project.
To put it in perspective, InnoDB pages only fill each data page 15/16 full anyway. So the overhead of the InnoDB page format is likely to be greater than the savings you could get from micro-optimizing boolean storage.
From what I understand, one should avoid nullable columns in databases whenever possible.
But, in what specific situations do nullable columns actually cause a significant performance drop?
In other words, when does null really hurt performance? (As opposed to when it's negligible, and does not matter at all).
I'm asking so I can know when and how it actually makes a difference.
Don't know where you heard it, but it's not true.
Nullable columns are there to represent data accurately: if a value is unknown, or not yet entered, NULL is a natural value to store. Null values are no more onerous to store or retrieve than values of any other type: most database servers store them in a single bit, which means that it will take less I/O and processor effort to retrieve a NULL value than assembling a varchar, BLOB, or text field from a bunch of fragments that may require walking through a linked list, or reading more disk blocks off the hard drive.
There are a couple instances marginally related to nullable columns that may affect performance:
If you create an index on a nullable column, and the actual values in the column are sparse (i.e. many rows have a NULL value, or only a very few values are present (as, with, say a controlled vocabulary value), the b-tree data structure used to index the column becomes much less efficient. Index traversals become more expensive operations when half the values in an index are identical: you end up with an unbalanced tree.
Inapt use of NULL values, or inappropriate query techniques that don't utilize NULL values as they were designed often results in poor performance, because progammers often fall back on the bad habit of searching or joining on computed column values, which ignores the fantastic set-processing ability of modern db servers. I've consulted at lots of places where the development staff has made a habit of writing clauses like:
WHERE ISNULL(myColumn, '') = ''
which means that the DB server cannot use an index directly, and must perform a computation on every single row of that section of the execution tree to evaluate the query. That is not because there is any intrinsic inefficiency in storing, comparing, or evaluating NULL values, but because the query thwarts the strengths of the database engine to achieve a particular result.
What is the difference between doing:
SELECT * FROM table WHERE column IS NULL
or -
SELECT * FROM table WHERE column = 0
Is doing IS NULL significantly worse off than equating to a constant?
The use case comes up where I have something like:
SELECT * FROM users WHERE paying IS NULL
(or adding an additional column)
SELECT * FROM users WHERE is_paying = 0
If I understand your question correctly, you are asking about the relative benefits/problems with the two situations:
where is_paying = 0
where paying is null
Given that both are in the data table, I cannot think of why one would perform better than the other. I do think the first is clearer on what the query is doing, so that is the version I would prefer. But from a performance perspective, they should be the same.
Someone else mentioned -- and I'm sure you are aware -- that NULL and 0 are different beasts. They can also behave differently in the optimization of joins and other elements. But, for simple filtering, I would expect them to have the same performance.
Well, there is one technicaility. The comparison to "0" is probably built into the CPU. The comparison to NULL is probably a bit operation that requires something like a mask, shift, and comparison -- which might take an iota of time longer. However, this performance difference is negligible when compared to the fact that you are reading the data from disk to begin with.
comparing to NULL and zero are two different things. zero is a value (known value) while NULL is UNKNOWN. The zero specifically means that the value was set to be zero; null means that the value was not set, or was set to null.
You'll get entirely different results using these queries, it's not simply a matter of performance.
Suppose you have a variety of users. Some have non-zero values for the "paying" column, some have 0, and some don't have a value whatsoever. The last case is what "null" more or less represents.
As for performance, do you have an index on the "paying" column? If you only have a few hundred rows in the table, this is probably irrelevant. If you have many thousands of rows, you are basically telling the query to iterate over every row of the table unless you have some indexing in place. This is true regardless of whether you are searching for "paying = 0" or "paying is null".
But again, just to reemphasize, the two queries will give you completely different results.
As far as I know comparing to NULL is as fast as comparing to 0, so you should choose based on:
Simplicity - use the option which makes your code simpler
Minimal size - use the option which makes your table smaller
In this case making the paying column NULL-able will probably be better.
You should also check out these questions:
NULL in MySQL (Performance & Storage)
MySQL: NULL vs “”
Is it better to use default null or default "" for text fields in MySQL?
Why?
Update: I know what means each of them. I am interested what is better to use considering disk space and performance.
Update 2: Hey ppl! The question was "what is better to use" not "what each means" or "how to check them"...
For MyISAM tables, NULL creates an extra bit for each NULLABLE column (the null bit) for each row. If the column is not NULLABLE, the extra bit of information is never needed. However, that is padded out to 8 bit bytes so you always gain 1 + mod 8 bytes for the count of NULLABLE columns. 1
Text columns are a little different from other datatypes. First, for "" the table entry holds the two byte length of the string followed by the bytes of the string and is a variant length structure. In the case of NULL, there's no need for the length information but it's included anyways as part of the column structure.
In InnoDB, NULLS take no space: They simply don't exist in the data set. The same is true for the empty string as the data offsets don't exist either. The only difference is that the NULLs will have the NULL bit set while the empty strings won't. 2
When the data is actually laid out on disk, NULL and '' take up EXACTLY THE SAME SPACE in both data types. However, when the value is searched, checking for NULL is slightly faster then checking for '' as you don't have to consider the data length in your calculations: you only check the null bit.
As a result of the NULL and '' space differences, NULL and '' have NO SIZE IMPACT unless the column is specified to be NULLable or not. If the column is NOT NULL, only in MyISAM tables will you see any peformance difference (and then, obviously, default NULL can't be used so it's a moot question).
The real question then boils down to the application interpretation of "no value set here" columns. If the "" is a valid value meaning "the user entered nothing here" or somesuch, then default NULL is preferable as you want to distinguish between NULL and "" when a record is entered that has no data in it.
Generally though, default is really only useful for refactoring a database, when new values need to come into effect on old data. In that case, again, the choice depends upon how the application data is interpreted. For some old data, NULL is perfectly appropriate and the best fit (the column didn't exist before so it has NULL value now!). For others, "" is more appropriate (often when the queries use SELECT * and NULL causes crash problems).
In ULTRA-GENERAL TERMS (and from a philosophical standpoint) default NULL for NULLABLE columns is preferred as it gives the best semantic interpretation of "No Value Specified".
1 [http://forge.mysql.com/wiki/MySQL_Internals_MyISAM]
2 [http://forge.mysql.com/wiki/MySQL_Internals_InnoDB]
Use default null. In SQL, null is very different from the empty string (""). The empty string specifically means that the value was set to be empty; null means that the value was not set, or was set to null. Different meanings, you see.
The different meanings and their different usages are why it's important to use each of them as appropriate; the amount of space potentially saved by using default null as opposed to default "" is so small that it approaches negligibility; however, the potential value of using the proper defaults as convention dictates is quite high.
From High Performance MySQL, 3rd Edition
Avoid NULL if possible.
A lot of tables include nullable columns even when the application does not need
to store NULL (the absence of a value), merely because it’s the default. It’s usually
best to specify columns as NOT NULL unless you intend to store NULL in them.
It’s harder for MySQL to optimize queries that refer to nullable columns, because
they make indexes, index statistics, and value comparisons more complicated. A
nullable column uses more storage space and requires special processing inside
MySQL. When a nullable column is indexed, it requires an extra byte per entry
and can even cause a fixed-size index (such as an index on a single integer column)
to be converted to a variable-sized one in MyISAM.
The performance improvement from changing NULL columns to NOT NULL is usually
small, so don’t make it a priority to find and change them on an existing schema
unless you know they are causing problems. However, if you’re planning to index
columns, avoid making them nullable if possible.
There are exceptions, of course. For example, it’s worth mentioning that InnoDB
stores NULL with a single bit, so it can be pretty space-efficient for sparsely populated
data. This doesn’t apply to MyISAM, though.
I found out that NULL vs "" is insignificant in terms of disk-space and performance.
The only true reason I can personally see in using NULL over '' is when you have a field marked as UNIQUE but need the ability to allow multiple "empty" columns.
For example, the email column in my user table is only filled in if someone actually has an email address. Anyone without an email address gets NULL. I can still make this field unique because NULL isn't counted as a value, whereas the empty string '' is.
A lot of folks are answering the what is the difference between null and '', but the OP has requested what takes up less space/is faster, so here's my stab at it:
The answer is that it depends. If your field is a char(10), it will always take 10 bytes if not set to null, and therefore, null will take up less space. Minute on a row-by-row basis, but over millions and millions of rows, this could add up. I believe even a varchar(10) will store one byte (\0) as an empty string, so again this could add up over huge tables.
In terms of performance in queries, null is in theory quicker to test, but I haven't seen able to come up with any appreciable difference on a well indexed table. Keep in mind though, that you may have to convert null to '' on the application side if this is the desired return. Again, row-by-row, the difference is minute, but it could potentially add up.
All in all it's a micro-optimization, so it boils down to preference. My preference is to use null because I like to know that there's no value there, and not guess if it's a blank string ('') or a bunch of spaces (' '). null is explicit in its nature. '' is not. Therefore, I go with null because I'm an explicit kind of guy.
Use whatever makes sense. NULL means "no value available/specified", "" means "empty string."
If you don't allow empty strings, but the user does not have to enter a value, then NULL makes sense. If you require a value, but it can be empty, NOT NULL and a value of "" makes sense.
And, of course, if you don't require a value, but an empty value can be specified, then NULL makes sense.
Looking at an efficiency point of view, an extra bit is used to determine whether the field is NULL or not, but don't bother about such micro-optimization until you have millions of rows.
"" is like an empty box... null is like no box at all.
It's a difficult concept to grasp initially, but as the answers here plainly state - there is a big difference.
'' = '' yields TRUE which satisfies WHERE condition
NULL = NULL yields NULL which doesn't satisfy WHERE condition
Which is better to use depends on what result you want to get.
If your values default to NULL, no query like this:
SELECT *
FROM mytable
WHERE col1 = ?
will ever return these values, even if you pass the NULL for the bound parameter, while this query:
SELECT *
FROM mytable
WHERE col1 = ''
will return you the rows that you set to an empty string.
This is true for MySQL, but not for Oracle, which does not distinguish between empty string and a NULL.
In Oracle, the latter query will never return anything.
Use "". It requires less programming effort if you can assert that columns are non-null. Space difference between these is trivial.
I prefer null when it is semantically correct. If there is an address field available and the user did not fill in, I give it a "". However if there in an address attribute to in the users table yet I did not offer the user a chance to fill it in, I give it a NULL.
I doubt (but I can't verify) that NULL and "" makes much of a difference.
In general, NULL should indicate data that is not present or has not been supplied, and therefore is a better default value than the empty string.
Sometimes the empty string is what you need as a data value, but it should almost never be a default value.
NULL means 'there is no value' and is treated especially by RDBMSs regarding where clauses and joins.
"" means 'empty string' and is not treated especially.
It depends on what does the text represent and how will it actually be used in queries.
For example, you can have a questionnaire with some obligatory questions and some optional questions.
Declined optional questions should have a NULL in their corresponding column.
Obligatory questions should have an empty string as default, because they HAVE to be answered. (Of course in a real application you'd tell the user to enter something, but I hope you get the idea)
Is there a reason for or against setting some fields as NULL or NOT NULL in a mysql table, apart from primary/foreign key fields?
That completely depends on your domain to be honest. Functionally it makes little difference to the database engine, but if you're looking to have a well defined domain it is often best to have both the database and application layer mirror the requirements you are placing on the user.
If it's moot to you whether or not the user enters their "Display Name", then by all means mark the column as nullable. On the other hand, if you are going to require a "Display Name" you should mark it non null in the database as well as enforcing the constraint in the application. By doubling the constraint, you ensure that should your front-end change, the domain is still fully qualified.
MySQL has a NOT NULL condition on a field, but this will not stop you from inserting "empty" data. There is no way to flag a field as "required".
As Pekka mentioned, you should be doing some sort of validation to prevent this at a higher level in your application.
It's not a MySQL specific thing - every database that I'm aware of allows for defining columns with a constraint that either allows a NULL value in the column, or does not allow this to happen.
Defining a column as NOT NULL means there always has to be a value present that matches the data type. NULL is a sentinel value, and its' data type transcends whatever is defined for the column.
If the column is a foreign key, the value also has to already exist in the related table before you insert the value into the current table. DEFAULT constraints are common, but not necessary, on columns defined as NOT NULL so that the columns will be populated with an appropriate value if NULL was attempted to be inserted into these columns. Getting back to foreign keys, a foreign key column can be nullable, which means the relationship is optional - the business rules allow for there to be no relationship.
When Should NULL & NOT NULL be Used?
Ideally, every column should be NOT NULL but it really depends on what the business rules require.
I don't know how you would define a required field in mySQL, care to enlighten me? I really don't know.
Anyway, even if this can be done, I can hardly think of a scenario where it would make sense. IMO, you would have to validate faulty (=incorrectly empty) data much earlier. Validation, sanitation and cutting should be done long before anything enters the database. The only time a database error should occur is when something exceptional occurs, e.g. when the database is physically not reachable.