This question deals with how one should handle conditional insertions into tables.
Suppose we customers and employees.
A customer can only be assigned 4 employees at a time.
We will come back to this in a moment.
On the database level, we have checks and triggers.
In MariaDB, CHECK constrains cannot have subqueries, so we cannot impose constraints
regarding degree of participation in this way. For instance, we cannot say something
like
CHECK ( customer_id IN (SUBQUERY that returns count of Employees with == 4 employees) ).
Triggers may be the solution.
insert-or-update). If a manager attempts to assign a customer to an employee who already has 4 customers, we should not allow that record to be inserted into the table that links customers to their employees. In this case we want a trigger that acts before insertion. We only want the
insertion to occur if that employee is not in the subquery that lists
employees with 4 customers.
We want to stop the insertion, but by law, the trigger will not do this
based on a condition that is merely stated in the trigger. From my understanding, the only way to do this is to send a signal (look at Use a trigger to stop an insert or update
This leads to my next two questions.
Is using a signal 'ideal'? Is it problematic? Is there a better way to insert into a table
based on a condition, instead of merely performing side effects prior to
an insertion or perhaps after an insertion?
It appears that the db would send a signal if the constraint was violated to begin with, so would this ever impact the application built on top of it?
Some flavors of restraint can be dealt with thus:
INSERT INTO TABLE (a,b,c)
SELECT ...
WHERE <-- put the logic here (if possible)
That is, arrange to have the SELECT deliver more or fewer rows based on your business logic.
Related
Currently we have a ticket management system and like all ticketing systems it needs to assign cases in a round-robin manner to the agents. Also, at the same time the agent can apply their own filtering logic and work on their queue.
The problem,
The table with the tickets is very large now, spans over 10 million rows.
One ticket should never be assigned to two different users.
To solve the above problem, this is the flow we have,
Select query is fired with filter criteria and limit 0,1
The row returned by the above query is then selected based on id and locked for update.
Lastly we fire the update saying user X has picked the case.
While step 3 executes other user cannot get a lock on the same case, so they fire 3.a query may be multiple times to get the next available case.
As number of users increase this time in step 4 goes higher and higher.
We tried doing a select for update in query at step 4 itself, but it makes the entire query slow. Assuming this is because a huge number of rows in the select query.
Questions,
Is there a different approach we need to take altogether?
Would doing a select and update in a stored procedure ensure the same results as doing a select for update and then update?
P.S - I have asked the same question stackexchange.
The problem is that you are trying to use MySQL level locking to ensure that a ticket cannot be assigned to more than one person. This way there is no way to detect if a ticket is locked by a user.
I would implement an application level lock by adding 2 lock related fields to the tickets table: a timestamp when the lock was applied and a user id field telling you which user holds the lock. The lock related fields may be held in another table (shopping cart, for example can be used for this purpose).
When a user selects a ticket, then you try to update these lock fields with a conditional update statement:
update tickets
set lock_time=now(), lock_user=...
where ticket_id=... and lock_time is null
Values in place of ... are supplied by your application. lock_time is null criteria is there to make sure that if the ticket has already been selected by another user, then the later user does not override the lock. After the update statement check out the number of rows affected. If it is one, then the current user acquired the lock. If it is 0, then someone else locked the ticket.
If you have the locking data in another table, then place a unique restriction on the ticket id field in that table and use insert to acquire a lock. If the insert succeeds, then the lock is acquired. If it fails, then another user has locked the ticket.
The lock is usually held for a number of minutes, after that your application must release the lock (set locking fields to null or delete the locking record from the other table).
I have a star schema that tracks Roles in a company, e.g. what dept the role is under, the employee assigned to the role, when they started, when/if they finished up and left.
I have two time dimensions, StartedDate & EndDate. While a role is active, the end date is null in the source system. In the star schema i set any null end dates to 31/12/2099, which is a dimension member i added manually.
Im working out the best way to update the Enddate for when a role finishes or an employee leaves.
Right now im:
Populating the fact table as normal, doing lookups on all dimensions.
i then do a lookup against the fact table to find duplicates, but not including the EndDate in this lookup. non matched rows are new and so inserted into the fact table.
matching rows then go into a conditional split to check if the currentEndDate is different from the newEnd Date. If different, they are inserted into an updateStaging table and a proc is run to update the fact table
Is there a more efficient or tidier way to do this?
How about putting all that in a foreach container, it would iterate through and be much more efficient.
I think it is a reasonable solution. I personally would use a Stored Proc instead for processing efficiency, but with your dimensional nature of the DWH and implied type 2 nature, this is a valid way to do it.
The other way, is to do your "no match" leg of the SSIS as is, but in your "match" leg, you could insert the row into the actual fact table, then have a post process T-SQL step which would update the two records needed.
I have a food_table and a person_table. Then I have a third table fav_food_table that stores the relation between food and person using food_id and person_id.
When the person goes to account info and updates his favourite food, the input data is passed to the PHP (HTTP) script as an array of selected food_id. A person can have multiple fav_food, the relation is one-to-many.
The naïve way to update fav_food_table is to delete from fav_food_table all that belongs to person_id then re-insert all the rows again. Thus, using 2 statements.
Is there a single statement that can do the same thing?
PSUEDO CODE:
CREATE TABLE food_table (food_id, food_name);
CREATE TABLE person_table (person_id, person_name);
CREATE TABLE fav_food (person_id, food_id);
You have a m:n relationship between person_table and food_table. This means you have multiple records in your relationship table related to the same person. When a person updates their favourite foods, any combination of four independent cases can occur:
Food A was favourite before, but is not anymore (DELETE FROM fav_food_table)
Food B was favourite before and still is! (do nothing)
Food C was no favourite before, but now is a favourite (INSERT INTO fav_food_table)
Food D was no favourite before, and still is no favourite! (do nothing)
To correctly keep your database up to date, you have to handle all four cases. Cases 2 and 4 are covered easily. Just don't do anything :)
That means, you have to do at least two steps to keep your database up to date: 1 and 3.
Your goal seems to be to reduce the number of sql statements, that have to be executed.
Deleting all favourites for one person from the table can always be done with a single statement:
DELETE
FROM fav_food_table
WHERE person_id = ?;
To delete selected favourites for one person, only an AND food_id IN (?,?,?) has to be added to the WHERE clause.
Inserting into the table can also be done with a single statement:
INSERT INTO fav_food_table (person_id,food_id)
VALUES (?,?),
(?,?),
(?,?),
.....;
Summary as of right now:
No matter, whether we delete all old records for this person and then insert all new records, or whether we only delete selected records and insert new ones: We can do it with two statements!
In the second case (the "smart" case), however, we need to know not only the new state of the relation, but also the old state to compute the difference between the two. This will result in either one more SELECT statement or needs some smart "client" (PHP) logic.
Your two step process doesn't seem to be naive, but easy and effective to me. There is no way to reduce this process to less than two statements.
If you want to reduce the number of times you have to effectively send commands from PHP to the server, you can either look into mysqli_multi_query() or into creating a stored procedure which holds both your DELETE and INSERT statement. But bottom line, this will be the same thing as executing the two queries on its own.
You can also look into MySQL Transactions to implement a safer process and be able to rollback your DELETE command should an error occur later on.
May I suggest something different. Instead of deleting, updating?
Deleting data in common is something that should be thought off. Deleting data cannot be retrieved. So check this out.
We are adjusting your code a bit.
CREATE TABLE food_table (food_id, food_name);
CREATE TABLE person_table (person_id, person_name);
CREATE TABLE fav_food (person_id, food_id, fav_food_active);
/* see the last column i added. It should be a bit type and can only hold the values 0 and 1. */
Now you technically can update this everytime. No need for deleting the values. This statement is
/* deletion */
UPDATE Fav_Food
SET fav_food_active = 0
WHERE food_id = (your food_id)
AND person_id = (your person_id)
/* activating it again */
UPDATE Fav_Food
SET fav_food_active = 1
WHERE food_id = (your food_id)
AND person_id = (your person_id)
So now you switch between those 2 for activating and deleting it, without having the consequences of deleting hard data. Overal, you can just call it like this in your code
SELECT *
FROM fav_food
WHERE (here your where clause on which you wanna search for)
AND fav_food_active = 1
Remember when you enter something in the database, you should always add it as 1. You can do that in PHP myadmin as auto value, or hard code it in your INSERT statement.
I am not sure what your backend code is (okay taking that back, its PHP, i read your post) looking at your question you shooting everything through an array, but try to work in some checks that foreach fav_food entry, check first in the db if the query excists. If it does, update it, if not, insert it. and let that run in a loop.
So something like
foreach ($food_id as $value){
// check here the overal statement if it excists in your db
// give back count
if ($count == 1){
// update query to delete it.
}else{
// create your insert query here
}
}
Hope this helps. Happy coding!
Hi I have started moving access into mysql and I was wondering if there is a constraint or something I could use to be able to make a column not null and still have empty values in it?
This is not my own database, if it was I would just fill in the empty fields and then change the column to not null.
Yes, there are various approaches for modelling missing information without using nulls.
You can choose a value to represent missing. It's quite hard to genrealize so here are a few examples. For the end_date attribute in an open-ended period (i.e. has started but is in progress and not yet finished), use a far-future date such as 9999-12-31. For a person_middle_name attributte, Joe Celko suggests placing metadata values in double-curly braces e.g. {{NK}} for 'not known', {{NA}} for 'not applicable', etc.
Another somewhat intuitive approach for modelling missing information is by the absence of a row in a table. If an employee is unsalaried then do not add a row for them in the Payroll table, thus making them distinct from a salaried employee who is currently receiving no salary represented by a salary_amount of zero in the Payroll table.
A further approach is by the presence of a row in a table. You could have tables for Salaried, Unsalaried and SalaryUnknown and ensure every employee has one row in exactly one of these tables (perhaps enforced in MySQL using triggers and/or procedures).
I have a table A (SQL Server 2008) that contains sets of min and max numbers. In one of my stored procedures I use this table joined with a product table B to find an available product number that's between min and max and then insert a new product with this product number. So I want the next free number that's between min/max.
Between finding out the next available number and inserting the product row I want a lock to prevent anyone of finding the same number (and giving a duplicate).
How should I think in this situation? Should I get an update lock on the A table even though I never modify it? The lock should be released after I do the insert into table B and the transaction finishes? Will this update lock prevent other transactions from reading table A?
Edit: The min/max table is a table for different product series. Depending on which serie you want i want to try and find an available number in this sequence. The productnr is not unique, it would be possible to make it unique in combination with a second column though. Simplified sp:
CREATE PROCEDURE [dbo].[InsertProduct]
(
#Param1 int,
#Param2 bit,
...
#Param20 int) AS
BEGIN
DECLARE #ProductNr int
--Here I do a query to determine which ProductNr I should have. Checking that the number is between max/min in the series and that no other product has this productnr.
--Now insert the row
INSERT INTO Products VALUES (#Param1, #ProductNr, ...., #Param2
END
Your question is kind of obscure, it would help if you included some sample data.
Regardless, a tactic that I have used in the past is to try and wrap everything withing a single statement -- here, it would be an INSERT. Every SQL statement is de-facto wrapped in its own implicit transaction (that's atomicity, what the "A" in the ACID of relational database properties fame stands for). In psuedo-code, it'd look something like:
INSERT MyTable (Id, Plus, Other, Columns)
select CalcForNewId, Plus, Other, Columns
from [what may be a pertty convoluted query to determine the "next" valid Id]
This only works if you can write your business logic as a single reasonable query (where "reasonable" means it doesn't lock, block, or deadlock either the current or any other user for an unreasonable length of time). That can be a pretty tall order, but I'd take that over having to write complex BEGIN TRANSACTION/COMMIT/ROLLBACK code intermixed with TRY...CATCH blocks any day. (That will of course work, and I've upvoted #Scott Bruns accordingly.)
Just insert the next number into table B. If it commits it is yours to use. If you get a Key Violation it means that a different process has just entered the new number. In that case increment the new number and try again.
The database will automatically handle the concerency for you. No manual locking is required.