I want to see if there is any way to do what I'm trying to do without using cursors, or any looping logic. I have these 2 tables. There is a ONE-MANY relationship between FileRunData and Invoice, and they're linked by RunId/FileRunId.
CREATE TABLE [dbo].[FileRunData](
[RunId] [uniqueidentifier] primary key,
[Status] [varchar](25) NOT NULL,
[IsEmailSent] [bit] NOT NULL
)
CREATE TABLE [dbo].[Invoice](
[FileRunId] [uniqueidentifier] NULL,
[InvoiceId] [uniqueidentifier] primary key,
[InvoiceType] [varchar](20) NOT NULL,
[Status] [varchar](25) NULL
)
I want to send an email notification for the following condition.
In the FileRunData if Status='Processed' and IsEmailSent=0, then I have to check all the rows in the Invoice table for that FileRunId, and if their Status is 'Invoiced', then I have to send an email.
My approach (will be using cursor here)
select RunIds from FileRunData
for each RunId, get the number of rows for that RunId from Invoice
get the number of rows for that RunId which have Status='Invoiced'
If both 3&4 are equal, then send the email
I think by adding a couple of negations in, the problem is simpler to understand, unless I've severely misunderstood you. We don't want to send an email if there's any row in Invoice which has a status other than Invoiced:
SELECT frd.RunId
FROM FileRunData frd
WHERE frd.Status = 'Processed' AND
frd.IsEmailSent = 0 AND
NOT EXISTS (select * from Invoice i
where i.FileRunID = frd.RunID and
i.Status != 'Invoiced')
should select the IDs for runs which meet your conditions. Unfortunately, if you need to use these RunId values and send one email for each such result, then unfortunately, at this point you'll still have to use a cursor1 to process this result set and make the actual sp_send_dbmail calls.
1Or any morally equivalent way of processing each row in turn. I'd normally just use a cursor but some people have an aversion to the word even appearing in their SQL and insist on creating temp tables and while loops instead, for example.
Related
In mysql how can I write a query that will fetch ALL business data, and at the same time (or not if it is better another way) check if user is following that business? I have the following relationship table to determine if a user is following a business (status=1 would mean that person is following):
CREATE TABLE IF NOT EXISTS `Relationship_User_Follows_Business` (
`user_id` int(10) unsigned NOT NULL,
`business_id` int(10) unsigned NOT NULL,
`status` tinyint(3) unsigned NOT NULL DEFAULT '0' COMMENT '1=following, 0=not following'
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
ALTER TABLE `Relationship_User_Follows_Business`
ADD UNIQUE KEY `unique_user_business_id` (`user_id`,`business_id`);
Assume business table just holds data on different businesses like name, phone number, etc. I would want to return all of the business data in my query (Business.*). I want to append the status (0 or 1) to the end of each business row to determine if the user is following that business. I have tried the following query but it does not work because it is narrowing the results to only show a business if there is a relationship row. I wish to show ALL businesses regardless if a relationship row exists or not because I only create the relationship row if a user follows:
SELECT Business.*, Relationship_User_Follows_Business.status FROM Business, Relationship_User_Follows_Business WHERE 104=Relationship_User_Follows_Business.user_id AND Business.id=Relationship_User_Follows_Business.business_id
Note that I am using 104 as a test user id. The user id would normally be dependent on user, not a static 104.
You are looking for a LEFT JOIN and not an INNER JOIN which keeps all the records from the master table and all the matching rows from the details table . Also, avoid using implicit join syntax(comma separated) and use the proper syntax of a join :
SELECT Business.*, Relationship_User_Follows_Business.status
FROM Business
LEFT JOIN Relationship_User_Follows_Business
ON Business.id = Relationship_User_Follows_Business.business_id
AND Relationship_User_Follows_Business.user_id = 104
I have two doubts:
I have a table as follows:
AUTHOR
(
authorID int NOT NULL,
authName varchar(255) NOT NULL,
authSurname varchar(255),
authPlaceOfBirth varchar(255),
authDOB date(),
authDoD varchar(255),
PRIMARY KEY (authorID)
)
Now, I want to find the authors who have died. That is, if the value of DoD exists in the table, then they have died. How to do this? That is, a particular value in a column exists?
Something like this:
SELECT authName
FROM AUTHOR
WHERE authDoD is not NULL?
Second, I have two tables as follows:
TABLE inventory_genre
{
genreID int NOT NULL,
inventoryID int NOT NULL,
PRIMARY KEY (genreID,inventoryID)
}
TABLE INVENTORY
{
inventoryID int NOT NULL,
title varchar(255),
wholesale int,
markup int,
qtyinStock int,
Discount int,
PRIMARY KEY (inventoryID)
}
I want to list all the genres that have no associated titles in the inventory. I know I have to subtract but I am not able to come up with it exactly. Please guide me in the right direction!
Not sure I understand the criteria you are describing in the first question, but either
select * from author where authDOB is not null;
or
select * from author where authDOB = 'some value that I dont know';
For the second one, you could use exists or in with a nested select:
select * from genre where id not in (select genreId from inventory_genre);
Part 1
Think about it logically:
you're looking for people who are referenced by what?
By their death.
How do you know they're dead?
Because a certain field in the table has been filled in.
Can you quantify this?
Yes, the value of their death exists.
So, yon can construct an SQL search that looks in the table of Authors for rows (people) where the column (authDoD) value is not nothing/null.
As a note for a perfect world you should have IS NULL set in the column authDoD so that if no value is set the column row value is NULL and easier to handle in queries.
SQL (assuming column can be empty but not Null):
SELECT * FROM authors WHERE authDoD > ''
SQL (assuming column can only be null unless dead):
SELECT * FROM authors WHERE authDoD IS NOT NULL
Part 2
You want to return a negative search, a search that turns up no results, so find each genre which does not feature in any inventory table row.
While I'm about to write a longer answer for this, the answer just posted by Tobb does exactly what you need.
Also note that you should ideally have an INDEX on your ID columns to make traversing them smoother and faster.
For the first part of your question, you could use
SELECT * FROM AUTHOR WHERE authDoD IS NOT NULL;
This would SELECT the deceased people
Your first problem is solved, if you check for authorDoD > ''. It will evaluate to false if there's NULL or the empty string ''.
To solve your second problem you could just JOIN your tables, since the inventoryID isn't allowed to be NULL.
JOIN INVENTORY i ON (inventory_genre.inventoryID = i.inventoryID)
After that you can check the existence of an title like above with title > ''.
I created a table with five columns:
userID,name,mobile_num,telphone_num,count
Either mobile or telephone number or both field must be filled in, and it should be unique. How can i create such a row in mysql?
CREATE TABLE information (
userID INT NOT NULL,
name varchar(255),
mobile_num INT UNIQUE,
telphone_num INT UNIQUE,
count INT,
PRIMARY KEY (userID)
)
the UNIQUE property forces the values to be unique, however that might be a problem for you because if two users GIVE you his mobile but DONT GIVE not their home phone, you won't be able to have two users with the value NULL, so I think the solution you are asking for might not be your best option.
The simplest way to do this is to enforce the requirement in the application code that inserts a row to the database.
This question suggests that it is not possible to enforce one of two columns not being null by a table constraint.
You could create an after insert trigger to check the requirements, but assuming the database is in the control of your application, just enforcing this requirement in the application makes most sense. The coding and error handling will be simpler.
It is not clear what you mean when you say the phone numbers "must be unique" but I would probably reconsider this requirement.
Two users could be at the same location (e.g. in the same household) and have the same phone number. So there could be duplicate phone numbers between users.
A user who uses their mobile number as their primary phone might enter the same phone number in both fields. I don't see any benefit in preventing this. Actually when this happens, you have more useful information than if you had prevented it. And trying to prevent it would probably result in a data entry interface that is confusing for the user.
CREATE TABLE information (
userID INT NOT NULL,
name varchar(255),
mobile_num INT,
telphone_num INT,
count INT,
PRIMARY KEY (userID,mobile_num,telephone_num)
)
Use the below query when face a situation to insert either any one number
insert into information(UserID,name,mobile_num,count)
values(3,'andrew',23443546,5);
or
insert into information(UserID,name,phone_num,count)
values(3,'andrew',23443546,5);
By default mysql will insert the value for other number as 0 in such situation,
So you can use 0 for the other number when you are going to insert only one number using
insert into information
values(3,'andrew',0,23443546,5);
insert into information
values(3,'andrew',23443546,0,5);
You can use a trigger:
DELIMITER //
CREATE TRIGGER checkPhoneNumbers
BEFORE INSERT ON tableName
FOR EACH ROW
BEGIN
SET #exists = (SELECT *
FROM tableName
WHERE mobile_num = NEW.mobile_num
|| mobile_num = NEW.telphone_num
|| telphone_num = NEW.telphone_num
|| telphone_num = NEW.telphone_num);
IF(#exists IS NOT NULL) THEN
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT = 'Telephone or mobile number already present!';
END IF;
END
//
DELIMITER ;
In this way you will have a db error if you insert a mobile or telephone number which is already present as mobile or telephone number
My questions comes first, then I'll describe the whole situation and current solution:
Questions.
1. Why could mySQL make enormously lots of continous read|write (300-1000 megaBytes) disk operations?
2. Is DB structure optimal (need advice otherwise)?
3. Do UniqueKey could slow down DB?
4. What could be better solution for the situation?
5. At the end vServer is getting down and I got mail with ~'ETIMEDOUT: Connection timed out - connect (2)'; So maybe issue is not in DB structure but it is some misconfiguration?
Situation.
Users on the end devices are playing and when gameover comes they are storing game records in central DB. Users could see highscores table sorted by hignscore.
I cant say that there are a lot of users. Lets assume that 1 user per 1 min.
Solution.
LAMP.
Since there are several similar games that users are playing there are several similar tables+views pairs in DB. (~25 Tables+25 Views total). Most of tables contain ~30 000 records. 3 of them contain up to 150 000 records.
In order to store users uniquely: 1user-1record I made a unique key UNIQUE INDEX userid (userid, gamename, gametype, recordvalue).
Since user should see sorted values (highscores) I made a view for a table that shows what is needed. So the external php script is working with view rather then with table.
CREATE TABLE supergameN (
id INT(11) NOT NULL AUTO_INCREMENT,
userid VARCHAR(255) NOT NULL,
username VARCHAR(50) NOT NULL,
gamename VARCHAR(100) NOT NULL,
gametype VARCHAR(100) NOT NULL,
description VARCHAR(100) NULL DEFAULT 'empty',
recordvalue INT(11) NOT NULL,
PRIMARY KEY (id),
UNIQUE INDEX userid (userid, gamename, gametype, recordvalue)
)
CREATE VIEW supergameN_view AS
SELECT
id,
userid,
username,
gamename,
gametype,
description,
recordvalue
FROM supergameN
ORDER BY gametype, recordvalue DESC
Thanks in advance. Alex.
Maybe not the solution but something I noticed:
Leave out the recordvalue from the unique key, since otherwise you would allow several records to exist for each userid-gamename-gametype combination, as long asd they have different recordvalues!
By using
UNIQUE INDEX userid (userid, gamename, gametype)
You ensure that per game and user you only ever store one result.
And, some further remarks/questions:
Do you really need two columns to identify the game?
What is kept in description: is it user or game related?
Maybe you could normalize a bit by having just a gameid column in your main table and (assuming that description refers to the game) a separate table games with columns gameid, gamename,gametypeand description. And then, of course, there would be no need to keep id anymore, instead you would have the combination of (userid,gameid) as your primary key.
The previous table this data was stored in approached 3-4gb, but the data wasn't compressed before/after storage. I'm not a DBA so I'm a little out of my depth with a good strategy.
The table is to log changes to a particular model in my application (user profiles), but with one tricky requirement: we should be able to fetch the state of a profile at any given date.
Data (single table):
id, username, email, first_name, last_name, website, avatar_url, address, city, zip, phone
The only two requirements:
be able to fetch a list of changes for a given model
be able to fetch state of model on a given date
Previously, all of the profile data was stored for a single change, even if only one column was changed. But to get a 'snapshot' for a particular date was easy enough.
My first couple of solutions in optimising the data structure:
(1) only store changed columns. This would drastically reduce data stored, but would make it quite complicated to get a snapshot of data. I'd have to merge all changes up to a given date (could be thousands), then apply that to a model. But that model couldn't be a fresh model (only changed data is stored). To do this, I'd have to first copy over all data from current profiles table, then to get snapshot apply changes to those base models.
(2) store whole of data, but convert to a compressed format like gzip or binary or whatnot. This would remove ability to query the data other than to obtain changes. I couldn't, for example, fetch all changes where email = ''. I would essentially have a single column with converted data, storing the whole of the profile.
Then, I would want to use relevant MySQL table options, like ARCHIVE to further reduce space.
So my question is, are there any other options which you feel are a better approach than 1/2 above, and, if not, which would be better?
First of all, I wouldn't worry at all about a 3GB table (unless it grew to this size in a very short period of time). MySQL can take it. Space shouldn't be a concern, keep in mind that a 500 GB hard disk costs about 4 man-hours (in my country).
That being said, in order to lower your storage requirements, create one table for each field of the table you want to monitor. Assuming a profile table like this:
CREATE TABLE profile (
profile_id INT PRIMARY KEY,
username VARCHAR(50),
email VARCHAR(50) -- and so on
);
... create two history tables:
CREATE TABLE profile_history_username (
profile_id INT NOT NULL,
username VARCHAR(50) NOT NULL, -- same type as profile.username
changedAt DATETIME NOT NULL,
PRIMARY KEY (profile_id, changedAt),
CONSTRAINT profile_id_username_fk
FOREIGN KEY profile_id_fkx (profile_id)
REFERENCES profile(profile_id)
);
CREATE TABLE profile_history_email (
profile_id INT NOT NULL,
email VARCHAR(50) NOT NULL, -- same type as profile.email
changedAt DATETIME NOT NULL,
PRIMARY KEY (profile_id, changedAt),
CONSTRAINT profile_id_fk
FOREIGN KEY profile_id_email_fkx (profile_id)
REFERENCES profile(profile_id)
);
Everytime you change one or more fields in profile, log the change in each relevant history table:
START TRANSACTION;
-- lock all tables
SELECT #now := NOW()
FROM profile
JOIN profile_history_email USING (profile_id)
WHERE profile_id = [a profile_id]
FOR UPDATE;
-- update main table, log change
UPDATE profile SET email = [new email] WHERE profile_id = [a profile_id];
INSERT INTO profile_history_email VALUES ([a profile_id], [new email], #now);
COMMIT;
You may also want to set appropriate AFTER triggers on profile so as to populate the history tables automatically.
Retrieving history information should be straightforward. In order to get the state of a profile at a given point in time, use this query:
SELECT
(
SELECT username FROM profile_history_username
WHERE profile_id = [a profile_id] AND changedAt = (
SELECT MAX(changedAt) FROM profile_history_username
WHERE profile_id = [a profile_id] AND changedAt <= [snapshot date]
)
) AS username,
(
SELECT email FROM profile_history_email
WHERE profile_id = [a profile_id] AND changedAt = (
SELECT MAX(changedAt) FROM profile_history_email
WHERE profile_id = [a profile_id] AND changedAt <= [snapshot date]
)
) AS email;
You can't compress the data without having to uncompress it in order to search it - which is going to severely damage the performance. If the data really is changing that often (i.e. more than an average of 20 times per record) then it would be more efficient to for storage and retrieval to structure it as a series of changes:
Consider:
CREATE TABLE profile (
id INT NOT NULL autoincrement,
PRIMARY KEY (id);
);
CREATE TABLE profile_data (
profile_id INT NOT NULL,
attr ENUM('username', 'email', 'first_name'
, 'last_name', 'website', 'avatar_url'
, 'address', 'city', 'zip', 'phone') NOT NULL,
value CARCHAR(255),
starttime DATETIME DEFAULT CURRENT_TIME,
endtime DATETIME,
PRIMARY KEY (profile_id, attr, starttime)
INDEX(profile_id),
FOREIGN KEY (profile_id) REFERENCES profile(id)
);
When you add a new value for an existing record, set an endtime in the masked record.
Then to get the value at a date $T:
SELECT p.id, attr, value
FROM profile p
INNER JOIN profile_date d
ON p.id=d.profile_id
WHERE $T>=starttime
AND $T<=IF(endtime IS NULL,$T, endtime);
Alternately just have a start time, and:
SELECT p.id, attr, value
FROM profile p
INNER JOIN profile_date d
ON p.id=d.profile_id
WHERE $T>=starttime
AND NOT EXISTS (SELECT 1
FROM prodile_data d2
WHERE d2.profile_id=d.profile_id
AND d2.attr=d.attr
AND d2.starttime>d.starttime
AND d2.starttime>$T);
(which will be even faster with the MAX concat trick).
But if the data is not changing with that frequency then keep it in the current structure.
You need a slow changing dimension:
i will do this only for e-mail and telephone so you understand (pay attention to the fact of i use two keys, 1 as unique in the table, and another that is unique to the user that it concerns. This is, the table key identifies the the record, and the user key identifies the user):
table_id, user_id, email, telephone, created_at,inactive_at,is_current
1, 1, mario#yahoo.it, 123456, 2012-01-02, , 2013-04-01, no
2, 2, erik#telecom.de, 123457, 2012-01-03, 2013-02-28, no
3, 3, vanessa#o2.de, 1234568, 2012-01-03, null, yes
4, 2, erik#telecom.de, 123459, 2012-02-28, null, yes
5, 1, super.mario#yahoo.it, 654321,2013-04-01, 2013-04-02, no
6, 1, super.mario#yahoo.it, 123456,2013-04-02, null, yes
most recent state of the database
select * from FooTable where inactive_at is null
or
select * from FooTable where is_current = 'yes'
All changes to mario (mario is user_id 1)
select * from FooTable where user_id = 1;
All changes between 1 jan 2013 and 1 of may 2013
select * from FooTable where created_at between '2013-01-01' and '2013-05-01';
and you need to compare with the old versions (with the help of a stored procedure, java or php code... you chose)
select * from FooTable where incative_at between '2013-01-01' and '2013-05-01';
if you want you can do a fancy sql statement
select f1.table_id, f1.user_id,
case when f1.email = f2.email then 'NO_CHANGE' else concat(f1.email , ' -> ', f2.email) end,
case when f1.phone = f2.phone then 'NO_CHANGE' else concat(f1.phone , ' -> ', f2.phone) end
from FooTable f1 inner join FooTable f2
on(f1.user_id = f2.user_id)
where f2.created_at in
(select max(f3.created_at) from Footable f3 where f3.user_id = f1.user_id
and f3.created_at < f1.created_at and f1.user_id=f3.user_id)
and f1.created_at between '2013-01-01' and '2013-05-01' ;
As you can see a juicy query, to compare the user_with the previews user row...
the state of the database on 2013-03-01
select * from FooTable where table_id in
(select max(table_id) from FooTable where inactive_at <= '2013-03-01' group by user_id
union
select id from FooTable where inactive_at is null group by user_id having count(table_id) =1 );
I think this is the easiest way of implement what you want... you could implement a multi-million tables relational model, but then it would be a pain in the arse to query it
Your database is not big enough, I work everyday with one even bigger. Now tell me is the money you save in a new server worthy the time you spend on a super-complex relational model?
BTW if the data changes too fast, this approach cannot be used...
BONUS: optimization:
create indexes on created_at, inactive_at, user_id and the pair
perform partition (both horizontal and vertical)
if you try and put all occurring changes in different tables and later if you require an instance on some date you join them along and display by comparing dates, for example if you want an instance at 1st of july you can run a query with condition where date is equal or less than 1st of july and order it in asc ordering limiting the count to 1. that way the joins will produce exactly the instance it was at 1st of july. in this manner you can even figure out the most frequently updated module.
also if you want to keep all the data flat try range partitioning on the basis of month that way mysql will handle it pretty easily.
Note: by date i mean storing unix timestamp of the date its pretty easier to compare.
I'll offer one more solution just for variety.
Schema
PROFILE
id INT PRIMARY KEY,
username VARCHAR(50) NOT NULL UNIQUE
PROFILE_ATTRIBUTE
id INT PRIMARY KEY,
profile_id INT NOT NULL FOREIGN KEY REFERENCES PROFILE (id),
attribute_name VARCHAR(50) NOT NULL,
attribute_value VARCHAR(255) NULL,
created_at DATETIME NOT NULL DEFAULT GETTIME(),
replaced_at DATETIME NULL
For all attributes you are tracking, simply add PROFILE_ATTRIBUTE records when they are updated, and mark the previous attribute record with the DATETIME it was replaced at.
Select Current Profile
SELECT *
FROM PROFILE p
LEFT JOIN PROFILE_ATTRIBUTE pa
ON p.id = pa.profile_id
WHERE p.username = 'username'
AND pa.replaced_at IS NULL
Select Profile At Date
SELECT *
FROM PROFILE p
LEFT JOIN PROFIILE_ATTRIBUTE pa
ON p.id = pa.profile_id
WHERE p.username = 'username'
AND pa.created_at < '2013-07-01'
AND '2013-07-01' <= IFNULL(pa.replaced_at, GETTIME())
When Updating Attributes
Insert the new attribute
Update the previous attribute's replaced_at value
It would probably be important that the created_at for a new attribute match the replaced_at for the corresponding old attribute. This would be so that there is an unbroken timeline of attribute values for a given attribute name.
Advantages
Simple two-table architecture (I personally don't like a table-per-field approach)
Can add additional attributes with no schema changes
Easily mapped into ORM systems, assuming an application lives on top of this database
Could easily see the history for a certain attribute_name over time.
Disadvantages
Integrity is not enforced. For example, the schema doesn't restrict on multiple NULL replaced_at records with the same attribute_name... perhaps this could be enforced with a two-column UNIQUE constraint
Let's say you add a new field in the future. Existing profiles would not select a value for the new field until they save a value to it. This is opposed to the value coming back as NULL if it were a column. This may or may not be an issue.
If you use this approach, be sure you have indexes on the created_at and replaced_at columns.
There may be other advantages or disadvantages. If commenters have input, I'll update this answer with more information.