I've to execute a complex query, selecting several columns from 7-8 tables.
We don't want to write that query in programming language (PHP - Symfony 1.4/Propel 1.4 in our case) but to create a view or stored procedure to have very simple select query for developers. I'm confused what will be better approach.
We need query in following format:
SET #PlayerId = 1;
SELECT CASE WHEN mat.player1id = #PlayerId THEN mat.player2id ELSE mat.player1id END as opponent
/*plus many other columns*/
FROM `wzo_matches` as mat /*plus few other tables*/
WHERE (mat.player1id =#PlayerId OR mat.player2id=#PlayerId)
/*plus other join conditions*/
Problem with view is, SET #PlayerId=xx statement. We don't know player id in advance but will be passed through PHP. I hope this is the reason to rule out views; is there any workaround for that?
Other option will be stored procedure. Only issue with that is, it will create a new view for every query so operation will be very heavy for DB.
Can someone suggest best approach so that developers can get required data from above query without writing above complex query in PHP. (Obviously through SP or view & simple select query from there)
Based on reply of Can I create view with parameter in MySQL?, My issue is fixed with following queries:
create function getPlayer() returns INTEGER DETERMINISTIC NO SQL return #getPlayer;
create view getPlay as
SELECT
CASE WHEN play.hiderid = getPlayer() THEN play.seekerid ELSE play.hiderid END AS opponent, play . *
FROM odd_play play, odd_match mat
WHERE (seekerid = getPlayer() OR hiderid = getPlayer())
AND play.id = mat.latestplay;
select play.*
from (select #getPlayer:=1 p) ply, getPlay play;
CREATE PROCEDURE SELECT_PLAYER(p INT) SET #PlayerId = p
SELECT CASE WHEN mat.player1id = #PlayerId THEN mat.player2id ELSE mat.player1id END as opponent
/*plus many other columns*/
FROM `wzo_matches` as mat /*plus few other tables*/
WHERE (mat.player1id =#PlayerId OR mat.player2id=#PlayerId)
/*plus other join conditions*/
Related
I hava three tables called t_asset,t_device and t_asset_device.The relationship between t_asset and the t_device is multiple pairs.Each table column is :
t_asset :id , asset_name,asset_code,create_time,creator
t_device:id, device_name,device_code,latitude,longitude,create_time,creator
t_assets_device:id,asset_id,device_id,create_time,creator
Now I want to get all the t_asset and the latitude,longitude of the first device,So I write the code and function like these:
fun_getLatitudeByAssetId(`assetId` varchar(50)){
BEGIN
declare v_latituede DECIMAL(10,5) DEFAULT(-1) ;
select latitude into v_latituede
from t_device tDevice
inner join t_assets_device tAssetsDevice
on tAssetsDevice.asset_id=assetId and
tDevice.id=tAssetsDevice.device_id
and tDevice.latitude!=-1
ORDER BY tDevice.id desc
limit 0,1;
return v_latituede;
END
}
fun_getLongititueByAssetId(`assetId` varchar(50)){
BEGIN
declare v_longititue DECIMAL(10,5) DEFAULT(-1) ;
select longititueinto v_longititue
from t_device tDevice
inner join t_assets_device tAssetsDevice
on tAssetsDevice.asset_id=assetId and
tDevice.id=tAssetsDevice.device_id
and tDevice.latitude!=-1
ORDER BY tDevice.id desc
limit 0,1;
return v_longititue ;
END
}
The final query sql is:
select tAsset.*,fun_getLatitudeByAssetId(tAsset.id) latitude,
fun_getLongititueByAssetId(tAsset.id) longititue from t_asset tAsset
It seems that I have query the latitude and longititue two times,If I want to get the other field from the t_device,I do not want to write another function
like fun_getDeviceCodeByAssetId, How can I optimized my code?
I don't think a function or procedure is the way to go - why not just define a view that has asset_id + all the other fields you want? Then just join to it on asset_id rather than calling functions. In addition to just being cleaner, I'd be concerned about performance with row rather than set processing with the function approach (this is total speculation, I don't have deep enough knowledge of MySQL to know how it's handled)
Is it really necessary to do it with functions?
You can do it with views, for example:
create view latitudeLongitude as
select latitude,longitude,asset_id
from t_device tDevice
inner join t_assets_device tAssetsDevice
on tDevice.id=tAssetsDevice.device_id
and tDevice.latitude!=-1;
Finally your last select should look like this:
select tAsset.*,latitudeLongitude.latitude,
latitudeLongitude.longititue
from t_assettAsset inner join latitudeLongitude
on t_assettAsset.id = latitudeLongitude.asset_id
If you're trying to return several values at once then you should rather declare a stored procedure, not a function. Then you'll be able to write select latitude, longitude from ... inside your procedure and then call it with a command like call getLatAndLong(...)
Hello guys.
I've an issue with a simple query.
Here we go, that's the code.
UPDATE user_resources AS ures
LEFT JOIN user_buildings as ub
ON ub.city_id = ures.city_id
INNER JOIN building_consumption AS bcons
ON bcons.resource_id = ures.resource_id
SET ures.quantity = ures.quantity - abs(FORMULA HERE that requires
building level and consumption at lvl 1 [default])
WHERE
(SELECT COUNT(id) FROM building_consumption AS bc2
WHERE bc2.building_id=ub.building_id) =
(SELECT COUNT(bc3.id) FROM building_consumption AS bc3
LEFT JOIN tmp_user_resources AS ures
ON ures.resource_id = bc3.resource_id
WHERE ures.city_id = ub.city_id
AND bc3.building_id=ub.building_id
AND bc3.quantity>0
AND IFNULL(ures.quantity, 0) - abs(FORMULA AGAIN);
I'll try to explain a bit.
As you can imagine, this is for a game.
Users (players) can has different buildings in different cities.
tab user_buildings
|id, city_id, buildings_id, level, usage|
A building can produce different resources
tab building_production
|id, building_id, resource_id, quantity_h|
but it can consume some resources too:
tab building_consumption
|id, building_id, resource_id, quantity_h|
Obviously a building cannot produce if there are not enough resources to consume for his job.
That's why I'm trying to compare WHERE SELECT COUNT how many resources it has to consume AND how many resources it can actually consume.
Mysql does NOT ALLOW to subquery same table inside an UPDATE stmt.
Using a cursor + loop is too much slow. I prefer to use different solution.
Temp table could be a solution but my problem now is how to update the temp table without triggers? (UPDATE + SELECT fires triggers and to avoid endless loops mysql block the query, and i can't pause/resume triggers because
IF ((#TRIGGER_CHECKS = FALSE)
OR (#TRIGGER_BEFORE_INSERT_CHECKS = FALSE))
AND (USER() = 'root#localhost')
THEN
LEAVE thisTrigger;
END IF;
is inside the trigger itself).
I am open to all your suggestions!
Thanks
P.S. The code must be inside a scheduled event.
I' m trying to write a stored procedure that will search a fairly simple database with
a USER table (user_id,name,...)
a USER_TYPE table (user_id,type_id) - multi to multi
a TYPE table (type_id,type_name)
a USER_GAME (user_id,game_id) -multi to multi
a GAME table (game_id,game_name)
A same user can have several games. Now, I want to be able to get the user according to a particular type and also according to a/some particular game(s), so that for example I can get all the user with, say type1, and with the games, say game2 and game5. I think I can get round the problem of several game names by passing them as a string parameter and do some kind of HAVING LIKE condition (I call get_user_spec('type1' , 'game3,game5') for example).
So far I get to that point:
CREATE DEFINER=`root`#`localhost` PROCEDURE `get_user_spec`(
IN inTypeName VARCHAR(50),
IN inGameName VARCHAR(150)
)
BEGIN
PREPARE statement FROM
"SELECT u.user_id,t.type_name,GROUP_CONCAT(g.game_name) AS game
FROM user u
INNER JOIN user_type ut
ON u.user_id=ut.user_id
INNER JOIN type t
ON ut.type_id=t.type_id
LEFT JOIN user_game ug
ON u.user_id=ug.user_id
LEFT JOIN game g
ON ug.game_id=g.game_id
WHERE t.type_name=?
GROUP BY u.user_id
HAVING game LIKE CONCAT('%',?,'%')
ORDER BY u.user_id";
SET #p1=inTypeName;
SET #p2=inGameName;
EXECUTE statement USING #p1,#p2;
END
But my real problem is that if I don't pass any game name, I then want to get all users with type1 (I then call get_user_spec('type1' , NULL). But I am then not getting anything as the procedure sees
HAVING game LIKE CONCAT('%',NULL,'%').
I hope that was clear enough. If anybody has any suggestions to get around that problem, I would be very grateful.
Thank you very much.
Change this line:
EXECUTE statement USING #p1,#p2;
to
EXECUTE statement USING #p1, ifnull(#p2, '');
This will cause the LIKE expression to be just '%%', which means "match everything"
I'm (attempting) to write a MySQL stored procedure that parses a large text file. Part of what this procedure does is check to see if the entities (in this case, government contractors) named in each record are already contained in the db. (This is a follow up to this question.) This is my first stored procedure and so I'm sure I've wondered off the rails here, and I would appreciated any help.
Here's what I have right now (after declaring the variables):
-- try and fetch first organization (a government agency)
SET agency = COALESCE(SELECT org_agency_o_id FROM orgs_agencies WHERE org_agency_code = maj_agency_cat,SELECT min(org_id) FROM orgs WHERE org_name LIKE CONCAT('U.S. ',SUBSTRING(maj_agency_cat,5)))
-- check to see if that worked
IF agency = NULL THEN
INSERT INTO orgs (org_name,org_name_length,org_type,org_sub_types) VALUES (CONCAT('U.S. ',SUBSTRING(maj_agency_cat,5)),LENGTH(CONCAT('U.S. ',SUBSTRING(maj_agency_cat,5))),'org','Org,GovernmentEntity,Federal,Agency');
SET agency = LAST_INSERT_ID();
END IF;
-- try and fetch second organization
SET org = COALESCE(SELECT MIN(org_id) FROM orgs WHERE org_name IN (vendorname, vendoralternatename, vendorlegalorganizationname, vendordoingasbusinessname), SELECT MIN(org_alias_org_id) FROM orgs_aliases WHERE org_alias in (endorname, vendoralternatename, vendorlegalorganizationname, vendordoingasbusinessname))
IF org = NULL THEN
INSERT INTO orgs(org_name,org_name_length,org_type,org_sub_types,org_created) VALUES (vendorname,LENGTH(vendorname),'org','org',DATE());
SET org = LAST_INSERT_ID();
END IF
Right now MySQL is throwing an error on the line:
SET agency = COALESCE(SELECT org_agency_o_id FROM orgs_agencies WHERE org_agency_code = maj_agency_cat,SELECT min(org_id) FROM orgs WHERE org_name LIKE CONCAT('U.S. ',SUBSTRING(maj_agency_cat,5)))
'maj_agency_cat' is a variable that I declare at the beginning of the procedure and then is assigned dynamically using a cursor that goes through my staging data. The full stored procedure can be viewed here.
I'm sure I'm missing something basic and would appreciate any help.
Try wrapping another () around the inner SELECT statements in your COALESCE arguments. Otherwise, they are not treated as subqueries to be executed first and the value returned, but as query objects passed into COALESCE, which is not a valid argument type for COALESCE:
SET agency = COALESCE((SELECT ..), (SELECT ..))
Im trying to make an operation of creating user network based on call detail records in my CDR table.
To make things simple lets say Ive got CDR table :
CDRid
UserAId
UserBId
there is more than 100 mln records so table is quite big.
I reated user2user table:
UserAId
UserBId
NumberOfConnections
then using curos I iterate through each row in the table, then I make select statement:
if in user2user table there is record which has UserAId = UserAId from CDR record and UserBId = UserBId from CDR record then increase NumberOfConnections.
otherwise insert such a row which NumebrOfConnections = 1.
Quite simple task and it works as I said using cursor but it is very bad in performance (estimated time at my computer ~60 h).
I heard about Sql Server Integration Services that it has got better performance when we are talking about such big tables.
Problem is that I have no idea how to customize SSIS package for creating such task.
If anyone has got any idea how to help me, any good resources etc I would be really thankful.
Maybe there is any other good solution to make it work faster. I used indexes and variable tables and so on and performance is still pure.
thanks for help,
P.S.
This is script which I wrote and execution of this takes sth like 40 - 50 h.
DECLARE CDR_cursor CURSOR FOR
SELECT CDRId, SubscriberAId, BNumber
FROM dbo.CDR
OPEN CDR_cursor;
FETCH NEXT FROM CDR_cursor
INTO #CdrId, #SubscriberAId, #BNumber;
WHILE ##FETCH_STATUS = 0
BEGIN
--here I check if there is a user with this number (Cause in CDR i only have SubscriberAId --and BNumber so that I need to check which one user is this (I only have users from
--network so that each time I cant find this user I add one which is outide network)
SELECT #UserBId = (Select UserID from dbo.Number where Number = #BNumber)
IF (#UserBId is NULL)
BEGIN
INSERT INTO dbo.[User] (ID, Marked, InNetwork)
VALUES (#OutUserId, 0, 0);
INSERT into dbo.[Number](Number, UserId) values (#BNumber, #OutUserId);
INSERT INTO dbo.User2User
VALUES (#SubscriberAId, #OutUserId, 1)
SET #OutUserId = #OutUserId - 1;
END
else
BEGIN
UPDATE dbo.User2User
SET NumberOfConnections = NumberOfConnections + 1
WHERE User1ID = #SubscriberAId AND User2ID = #UserBId
-- Insert the row if the UPDATE statement failed.
if(##ROWCOUNT = 0)
BEGIN
INSERT INTO dbo.User2User
VALUES (#SubscriberAId, #UserBId, 1)
END
END
SET #Counter = #Counter + 1;
if((#Counter % 100000) = 0)
BEGIN
PRINT Cast (#Counter as NVarchar(12));
END
FETCH NEXT FROM CDR_cursor
INTO #CdrId, #SubscriberAId, #BNumber;
END
CLOSE CDR_cursor;
DEALLOCATE CDR_cursor;
The thing about SSIS is that it probably won't be much faster than a cursor. It's pretty much doing the same thing: reading the table record by record, processing the record and then moving to the next one. There are some advanced techniques in SSIS like sharding the data input that will help if you have heavy duty hardware, but without that it's going to be pretty slow.
A better solution would be to write an INSERT and an UPDATE statement that will give you what you want. With that you'll be better able to take advantage of indices on the database. They would look something like:
WITH SummaryCDR AS (UserAId, UserBId, Conns) AS
(
SELECT UserAId, UserBId, COUNT(1) FROM CDR
GROUP BY UserAId, UserBId)
UPDATE user2user
SET NumberOfConnections = NumberOfConnections + SummaryCDR.Conns
FROM SummaryCDR
WHERE SummaryCDR.UserAId = user2user.UserAId
AND SummaryCDR.UserBId = user2user.UserBId
INSERT INTO user2user (UserAId, UserBId, NumberOfConnections)
SELECT CDR.UserAId, CDR.UserBId, Count(1)
FROM CDR
LEFT OUTER JOIN user2user
ON user2user.UserAId = CDR.UserAId
AND user2user.UserBId = CDR.UserBId
WHERE user2user.UserAId IS NULL
GROUP BY CDR.UserAId, CDR.UserBId
(NB: I don't have time to test this code, you'll have to debug it yourself)
is this what you need?
select
UserAId, UserBId, count(CDRid) as count_connections
from cdr
group by UserAId, UserBId
Could you break the conditional update/insert into two separate statements and get rid of the cursor?
Do the INSERT for all the NULL rows and the UPDATE for all the NOT NULL rows.
Why are you even considering doing row-by-row processing on a table that size? You know you can use the merge statment and insert or update and it will be faster. Or you could write an update to insert all rows that need updating in one set-based stament and an insert to insert alll rows when the row doesn't exist in one set-based statement.
Stop using the values clause and use an insert with joins instead. Same thing with updates. If you need extra complexity the case stamenet will probably give you all you need.
In general stop thinking of row-by-row processing. If you can write a select for the cursor, you can write a set-based statement to do the work 99.9% of the time.
You may still want a cursor with a table this large but one to process batches of data (for instance a 1000 records at time) not one to run ro-by-row.