I have an existing longtext column in a huge table (100MM rows) which includes json strings. Changing the type to json would lock the table. As an alternative I thought I could add json validation to the longtext column on create and update.
I do not have much experience with triggers. how would that work? I know it should be something like this:
TRIGGER `before_insert_user`
BEFORE INSERT
ON `users`
FOR EACH ROW
BEGIN
...
END;
TRIGGER `before_update_user`
BEFORE UPDATE
ON `users`
FOR EACH ROW
BEGIN
...
END;
Demo:
mysql> create table users (id serial primary key, properties longtext);
mysql> delimiter //
mysql> create trigger before_insert_user before insert on users for each row
-> begin
-> if not json_type(NEW.properties) = 'OBJECT' then
-> signal sqlstate '45000' set message_text = 'properties must be a valid JSON object';
-> end if;
-> end//
mysql> delimiter ;
mysql> insert into users set properties = 'yadda yadda';
ERROR 3141 (22032): Invalid JSON text in argument 1 to function json_type: "Invalid value." at position 0.
mysql> insert into users set properties = '["yadda yadda"]';
ERROR 1644 (45000): properties must be a valid JSON object
mysql> insert into users set properties = '{"yadda": "yadda"}';
Query OK, 1 row affected (0.01 sec)
However, I would change the data type to JSON. I would use pt-online-schema-change to make the alteration without locking the table.
That example at least ensures the longtext column is valid JSON format, and also is a JSON object (not an array or a scalar).
If you need the JSON to comply with a more precise format, read about JSON_SCHEMA_VALID().
Related
I have a SQL table that can reference another record in the table as its parent but should not reference itself. I have attempted to enforce this with a CHECK constraint but my attempts have failed as the id is an auto-increment column. Is there any other way to ensure that parent_id <> id?
My current attempt, which fails with error Check constraint 'not_own_parent' cannot refer to an auto-increment column. (errno 3818):
CREATE TABLE `content` (
`id` serial PRIMARY KEY NOT NULL,
`item_id` int NOT NULL,
`nested_item_id` int,
`block_id` int,
`order` int NOT NULL,
CONSTRAINT not_own_parent CHECK (nested_item_id <> id)
);
Here's a demo of using a trigger to cancel an insert that violates the condition you describe. You must use an AFTER trigger because in a BEFORE trigger the auto-increment value has not yet been generated.
mysql> delimiter ;;
mysql> create trigger t after insert on content
-> for each row begin
-> if NEW.nested_item_id = NEW.id then
-> signal sqlstate '45000' set message_text = 'content cannot reference itself';
-> end if;
-> end;;
mysql> delimiter ;
mysql> insert into content set item_id = 1, nested_item_id = 1, `order` = 1;
ERROR 1644 (45000): content cannot reference itself
mysql> insert into content set item_id = 1, nested_item_id = 2, `order` = 1;
Query OK, 1 row affected (0.01 sec)
Don't put this kind of thing in a constraint. For one thing, you can't do it directly in MySql. You'd have to use a trigger or something.
Instead:
write your CRUD code carefully, so it avoids generating incorrect rows. You have to do that anyway.
write a little program called "database_consistent" or something. Have it run a bunch of queries looking for any errors like the one you're trying to avoid. Have it send emails or SMSs if it finds problems. Run it often during development and at least daily in production.
One way to control auto-generated live values is by using triggers to manage new values.
For example, create instead of insert trigger to control newly generated ID. In triggers, you can make decisions based on the new value.
This is something that I stumbled across multiple projects, and I feel I'm reinventing the wheel every time:
I have a table that stores user data. Whenever the user is created, I create one row on the table. This row has several NULL fields on creation, because the user just informed some critical information (and other non-critical info is going to be filled later).
But, when the user completes the filling of the data, I want to enforce this cols to be NOT NULL.
Is there any way to make a column NULL for INSERT, but NOT NULL for UPDATE that does not involves triggers? Or any other clever solution?
Thank you
CREATE TABLE users ( regular_column SOMETYPE NOT NULL,
smart_column SOMETYPE NULL,
completed ENUM('no', 'yes') NOT NULL DEFAULT 'no',
CHECK ((completed = 'no') OR (smart_column IS NOT NULL)) );
This row has several NULL fields on creation, because the user just informed some critical information (and other non-critical info is going to be filled later).
In this moment completed = 'no', CHECK constraint is TRUE, smart_column may be NULL.
when the user completes the filling of the data, I want to enforce this cols to be NOT NULL.
In this moment completed = 'yes', and CHECK constraint does not allow NULL value in smart_column.
I.e. setting completed column to 'yes' fixes smart_column - you may alter it but cannot set it to NULL. And you cannot set completed to 'yes' until smart_column is set to a value.
If you use column options in your CREATE TABLE statement, there's no way a column can be both NULLable and NOT NULL. There's no way to distinguish between inserts and updates.
The alternative could be to let the column be nullable, but add a trigger on UPDATE that throws an exception (called a SIGNAL in MySQL terminology) if the column is still NULL after the update.
Here's a quick demo:
mysql> create table mytable (id int primary key, x int);
Query OK, 0 rows affected (0.05 sec)
mysql> delimiter //
mysql> create trigger notnullonupdate before update on mytable
-> for each row
-> begin
-> if NEW.x IS NULL then
-> signal sqlstate '45000'
-> set message_text = 'x must be updated to non-NULL value';
-> end if;
-> end//
mysql> delimiter ;
mysql> insert into mytable set id = 42, x = null;
Query OK, 1 row affected (0.03 sec)
mysql> update mytable set id = 42;
ERROR 1644 (45000): x must be updated to non-NULL value
Even though this is possible, it's kind of a lot of work.
Most developers would just handle this by writing application code to ensure the value is not null before executing an update. Of course the risk of that is if you forget one of the cases in your app that does an update, or if someone does an ad hoc update in the mysql client, it could cause your data to be in an invalid state and you wouldn't know it.
hello I have a datetime column and I would like to put a time restriction on it how would I do this?
For example a range of time from 3:00:00 to 15:00:00 all data that fits this criteria is stored if not throw and error up and stop the entering of data in the column
In MySQL, you'd have to do this with a trigger on INSERT and UPDATE, so if someone tries to enter a value that doesn't meet your criteria, you raise a SIGNAL.
mysql> CREATE TABLE MyTable (
my_datetime DATETIME
);
mysql> DELIMITER ;;
mysql> CREATE TRIGGER MyTable_ins BEFORE INSERT ON MyTable
FOR EACH ROW BEGIN
IF (NOT TIME(NEW.my_datetime) BETWEEN '03:00:00' AND '15:00:00') THEN
SIGNAL SQLSTATE '45000'
SET MESSAGE_TEXT = 'Time does not fall in the range allowed.'
END IF;
END;;
mysql> DELIMITER ;
I get the error if try to do something I shouldn't:
mysql> INSERT INTO MyTable SET my_datetime = '2017-01-13 18:00:00';
ERROR 1644 (45000): time does not fall in the range allowed
But it works if I choose a time that's allowed:
mysql> INSERT INTO MyTable SET my_datetime = '2017-01-13 11:00:00';
Query OK, 1 row affected (0.00 sec)
I did some digging and some reading. I tested some stuff out on my own server. It doesn't work. Then I found this answer:
CHECK constraint in MySQL is not working
Yep, it accepts a CHECK constraint as valid syntax, then completely ignores it.
And while I was testing and writing up, Bill has posted the correct answer for MySQL. Do what he says.
I'd like to make a TIMESTAMP field DEFAULT CURRENT_TIMESTAMP, for 'creation time' purpose. But if someone or somehow something changes that TIMESTAMP, my data won't be consistent.
Is there a way I can ensure it won't change unless I delete the row and reinsert it, other than application level?
With the suggested answer provided, i could work around with something like this
CREATE TRIGGER consistency1 BEFORE UPDATE ON table1
FOR EACH ROW
BEGIN
IF NEW.creationtime != OLD.creationtime THEN
SET NEW.creationtime = OLD.creationtime;
END IF;
END;
Since my comment has been appreciated, here's the extended version.
I personally don't think that it's possible.
Anyway, there are a couple of things you can try:
Make sure that only your application can write on the database
Write a trigger like this (pseudocode!)
create trigger prevent_change_timestamp on tbl_name
before update
#fetch old row value
#verify if the timestamp field has been changed
#raise an error (any SQL error will do)
Or like this
create trigger revert_change_timestamp on tbl_name
after update
#fetch pre-change row value
#update the row with the "old" value in place of the new one
I'd personally go with the 3rd option, if possible. Anyway, the 2nd one is good too. I'd not rely on the 1st option unless necessary (eg: no access to trigger functionality)
More info here: reference
It's funny in a way that database apps don't offer this functionality as standard: not only for a "created" timestamp field, but for things like autoincrement id fields, and any miscellaneous values which you may want to set on creating a record and then never allow to be changed... wonder what the rationale is?
What you can do here is, you can write a TRIGGER on the table when a row is being updated. In that trigger, you can compare the old and new values, and if they are different then you can just overwrite the new value with the old one.
I tried this in MySQL 5.1 and got an error
DELIMITER //
CREATE TRIGGER member_update_0
-> AFTER UPDATE ON members
-> FOR EACH ROW
-> BEGIN
-> IF NEW.id != OLD.id THEN
-> SET NEW.id = OLD.id;
-> END IF;
-> END;//
ERROR 1362 (HY000): Updating of NEW row is not allowed in after trigger
The same trigger with AFTER replaced by BEFORE is accepted;
to me, this is a counter-intuitive way to do it, but it works
delimiter ;
UPDATE members SET id=11353 WHERE id=1353;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 1 Changed: 0 Warnings: 0
It is actually possible to do this very neatly if you are using InnoDB.
Create another table with just one column. That column should have a foreign key (hence the innodb requirement in this solution) that points to the immutable column of the original table in question.
Put a restriction like "ON UPDATE RESTRICT".
In summary:
CREATE TABLE original (
....
immutable_column ...
INDEX index1(immutable_column)
....
) ENGINE=INNODB;
CREATE TABLE restricter (
.....
col1,
INDEX index2(col1),
FOREIGN KEY (col1) REFERENCES original (immutable_colum) ON UPDATE RESTRICT ON DELETE CASCADE
) ENGINE=INNODB;
Taking the idea a step further (for those of us still stuck with a legacy version of MySQL) you can have BOTH a protected & defaulted create_stamp AND an auto-updating update_stamp as follows:
If you have a table such as
CREATE TABLE `csv_status` (
`id` int(11) NOT NULL primary key AUTO_INCREMENT,
`create_stamp` datetime not null,
`update_stamp` timestamp default current_timestamp on update current_timestamp,
`status` enum('happy','sad') not null default 'happy'
);
Then you can define these triggers on it
drop trigger if exists set_create_stamp ;
create definer = CURRENT_USER trigger set_create_stamp BEFORE INSERT on
csv_status for each row
set NEW.create_stamp = now();
drop trigger if exists protect_create_stamp ;
delimiter //
create definer = CURRENT_USER trigger protect_create_stamp BEFORE UPDATE on
csv_status for each row
begin
if NEW.create_stamp != OLD.create_stamp then
set NEW.create_stamp = OLD.create_stamp;
end if;
end;//
delimiter ;
In MySQL I have a table with Column1 as NOT NULL:
create table myTable
(
Column1 int not null,
Column2 int not null
)
I can still insert an empty value like this:
INSERT INTO `myTable` ( `Column1` , `Column2` )
VALUES ( '66', '' );
How can I make the MySQL column also disallow blankstring?
EMPTY STRINGS
In ORACLE an empty string is used to represent NULL. In virtually everything else, however, an empty string is still a string, and so not NULL.
INTS
In your case you're actually inserting STRINGS into an INT column. This forces an implicit CAST operation.
When your RDBMS is converting the string '' to an INT it must get the value 0. As 0 is not NULL, this gets inserted.
A more valid test would be:
INSERT INTO `plekz`.`countries` (`Column1 ` , `Column2`)
VALUES (66, NULL);
EDIT
Sorry, I only half read your question. You also ask how to stop '' being inserted.
Your first problem is that you're inserting STRINGS and the table is defined as having INT fields. You can put constraints on the data that gets inserted, but these constraints will apply the the value after an conversion to an INT. Unless you want to prevent the value 0 from also being inserted, there is nothing you can do to the table to prevent this scenario.
Your better bet is to address why you are inserting strings in the first place. You could use a stored procedure that takes, and checks, the strings before converting them to INTs and then inserting them. Or, better still, you could make the checks in your client application.
A technically available option is to make the fields CHAR fields, then put a constraint on the fields, preventing '' from being inserted. I would strongly recommend against this.
You're inserting an empty string, not NULL. The constraint is only against NULL values, and it would appear that your database is not coercing empty strings to NULL when it converts them to INT (which raises the additional question of why you're inserting string literals into INT columns...)
MySQL, how to disallow empty string:
Create your table:
mysql> create table yar (val VARCHAR(25) not null);
Query OK, 0 rows affected (0.02 sec)
Create your 'before insert' trigger to check for blankstring and disallow.
mysql> create trigger foo before insert on yar
-> for each row
-> begin
-> if new.val = '' then
-> signal sqlstate '45000';
-> end if;
-> end;$$
Query OK, 0 rows affected (0.01 sec)
Try to insert null and blankstring into your column:
mysql> delimiter ;
mysql> insert into yar values("");
ERROR 1644 (45000): Unhandled user-defined exception condition
mysql> insert into yar values(NULL);
ERROR 1048 (23000): Column 'val' cannot be null
mysql> insert into yar values ("abc");
Query OK, 1 row affected (0.01 sec)
mysql> select * from yar;
+-----+
| val |
+-----+
| abc |
+-----+
1 row in set (0.00 sec)
Finally, Grumble to self and smack the nearest person who was responsible for picking mysql over postgresql.
As Martin mentions, depends on your RDBMS. Oracle treats empty strings as NULLs while others do not. See this SO post.
NULL is not equal to emptiness. In MySQL, there is an additional byte with each column entry to hold the "is null" information. To save space, a column is often defined as "not null" to spare this extra byte if the null status doesn't add any thing to the data model.