Hello to the lovely stack overflow community.
I have a set of values x values and I am trying to find the nearest larger value, then divide the difference by 2 and set this as a new value in new column called nearest_x. I have created a procedure with a while loop. The procedure runs and then give me no results. I think this is because I using the ele_id in the while loop, I think I need to simply look at each row in turn? Thoughts much appreciated. My first stack overflow post!
DELIMITER ;;
CREATE PROCEDURE highX_rowperrow()
BEGIN
DECLARE n INT DEFAULT 0;
DECLARE i INT DEFAULT 0;
SET n = (SELECT COUNT(*) FROM table_csa WHERE condition = "pal0");
SET i=0;
WHILE i<n DO #start while loop
SET #x1pal0 = (SELECT x FROM table_csa WHERE ele_id=(21001+i) AND condition = "pal0"); #count applicable rows (pal10 always first)
SET #Xnearest = (select x from table_csa where x > #x1pal0 and condition = "pal0" order by x asc limit 1); #Select the nearest larger than x value
SET #polypoint = #x1pal0+((abs(#x1pal)-abs(#Xnearest))/2); #calculate the difference and divide by 2 to set up point
UPDATE table_csa SET nearest_x = #polypoint WHERE ele_id=(21001+i); #put value in table
SET i=i+1;
END WHILE;
END ;;
DELIMITER ;
So grateful for any help, I thought this would be a simple obvious answer and I missed something as I'm learning. I guess not so simple, so I'm adding more information to make it more of a re-producible problem. I have quite a big data set so not immediately obvious how to do that but here is my effort. I also tried to explain the background so it makes more sense.
I have a table which has the following columns:
"ele_id" which is a set of location ids not necessarily individual in
numbers, but does generally count up by 1.
"x", which is x location,
"y" which is y location,
"condition" which is condition code for the location. So a different
condition codes do exist for the same location later on in the data
set.
"nearest_x" which is the column I want to populate
I also added a snapshot with example data
snapshot of table with example data
I am actually trying to turn these points into zones which I can make into a polygon, I have another data set in my database where I want to find which pieces of data fall within my polygons from the csa table. So I am trying to find the nearest x point value and calculate a point that falls half way between them, then I can do the same for y and draw a polygon. I know the condition pal0 has each location I need in it as an individual value which seemed like a good starting place.
As part of debugging I tried the following, this set of code performs what I want, but only on one element 21003 in this case, I can't seem to get it working in a while loop:
Set #var=2;
SET #x1pal0 = (SELECT x FROM table_csa WHERE ele_id=(21001+#var) AND condition = "pal0");
SET #Xnearest = (select x from table_csa where x > #x1pal0 and condition = "pal0" order by x asc limit 1);
SET #polypoint =#x1pal0+abs(#x1pal0-#Xnearest)/2;
Select #x1pal0, #Xnearest, #polypoint, ele_id from table_csa where ele_id=(21001+#var) and condition = "pal0" ;
UPDATE table_csa SET nearest_x = #polypoint WHERE ele_id=(21001+#var);
I needed to use a cursor not a while loop!
[MySQL_tutorial_link]
Related
si have a db field that i want to use to track intervals. i want to push completed intervals onto the db field when they are completed. to wit:
intervals = '10'
intervals = '1020' <-- pushing 20 onto the field
intervals = '102040' <-- pushing 40 onto the field
intervals = '102040SP' <-- pushing SP onto the field
the values will never decrement (and order doesn't really matter, if that's a factor), so i'm only looking for a way to UPDATE the field, but i have no idea how to do that because UPDATE tbl SET ... just overwrites the existing contents. i looked into CONCAT, but that works with variables the user provides, not with existing data AND additional user data. if i were to write some PSEUDO code, it might look like this:
UPDATE tbl PUSHTO intervals VALUE newInterval WHERE id='id' AND date='date'
so. can anybody help me out here? there has to be a way to do this. :)
An update with concatenation is what you want here:
UPDATE tbl
SET interval = CONCAT(interval, newInterval)
WHERE id = 'id' AND date = 'date';
If you wanted to make the update even in the event that newInterval might be null, you could try:
UPDATE tbl
SET interval = CONCAT(interval, COALESCE(newInterval, ''))
WHERE id = 'id' AND date = 'date';
I currently have an oracle table (lovalarm) containing around 600,000 rows. I need to be able to run a query which will cycle through each row and update a field (lovsiteid) to a random number between 14300 and 17300.
So far I have:
update lovalarm
set lovsiteid = (select TRUNC(dbms_random.value(14300,17300)) FROM dual)
Sadly this picks a random number and then updates all rows with the same number which isn't exactly what I'm after!
Can anyone point me in the right direction?
Many thanks,
Cap
Just not use subquery:
update lovalarm
set lovsiteid = TRUNC(dbms_random.value(14300,17300))
Try this:
update lovalarm set lovsiteid = (select FLOOR(RAND() * (17300 - 14300) + 14300))
works in MySQL
SQL:
$mysqli->query("UPDATE results
SET result_value = '".$row[0]['logo_value']."'
WHERE logo_id = '".$mysqli->real_escape_string($_GET['logo_id'])."'
AND user_id = '".$user_data[0]['user_id']."'");
This results table also contains result_tries I'd like to fetch before doing update, so I can use it to modify result_value... Is there a way to do it in a single shot instead of first doing select and than doing update?
Is this possible?
Basically:
UPDATE results SET result_value = result_value + $row[0][logo_value]
for just a simple addition. You CAN use existing fields in the record being updated as part of the update, so if you don't want just addition, there's not too many limits on what logic you can use instead of just x = x + y.
The query:
$consulta = "UPDATE `list`
SET `pos` = $pos
WHERE `id_item` IN (SELECT id_item
FROM lists
WHERE pos = '$item'
ORDER BY pos DESC
LIMIT 1)
AND id_usuario = '$us'
AND id_list = '$id_pl'";
The thing is, this query is inside a foreach, and it wants to update the order of the items in a list. Before I had it like this:
$consulta = "UPDATE `list`
SET `pos` = $pos
WHERE `$pos` = '$item'
AND id_usuario = '$us'
AND id_list = '$id_pl'";
But when I update pos 2 -> 1, and then 1 -> 2, the result is two times 2 and no 1...
Is there a solution for this query?
Renumbering the items in a list is tricky. When you renumber the items in the list using multiple separate SQL statements, it is even trickier.
Your inner sub-select statement also is not properly constrained. You need an extra condition such as:
AND id_list = '$id_pl'
There are probably many ways to do this, but the one that may be simplest follows. I'm assuming that:
the unshown foreach loop generates $pos values in the desired sequence (1, 2, ...)
the value of $id_pl is constant for the loop
the foreach loop gives values for $us and $item for each iteration
the combination of $id_pl, $us, and $item uniquely identifies a row in the list table
there aren't more than 100 pos values to worry about
you are able to use an explicit transaction around the statement sequence
The suggested solution has two stages:
Allocate 100 + pos to each row to place it in its new position
Subtract 100 from each pos
This technique avoids any complicated issues about whether rows that have had there position adjusted are reread by the same query.
Inside the loop:
foreach ...
...$pos, $item, $us...
UPDATE list
SET pos = $pos + 100
WHERE id_item = '$item'
AND id_usuario = '$us'
AND id_list = '$id_pl'
AND pos < 100
end foreach
UPDATE list
SET pos = pos - 100
WHERE id__list = '$id_pl';
If you don't know the size of the lists, you could assign negative pos values in the loop and convert to positive after the loop, or any of a number of other equivalent mappings. The key is to update the table so that the new pos numbers in the loop are disjoint from the old numbers, and then adjust the new values after the loop.
Alternative techniques create a temporary table that maps the old numbers to the new and then executes a single UPDATE statement that changes the old pos value to the new for all rows in a single operation. This is probably more efficient, especially if the mapping table can be generated as a query, but that depends on whether the renumbering is algorithmic. The technique shown, albeit somewhat clumsy, can be made to work for arbitrary renumberings.
Im trying to make an operation of creating user network based on call detail records in my CDR table.
To make things simple lets say Ive got CDR table :
CDRid
UserAId
UserBId
there is more than 100 mln records so table is quite big.
I reated user2user table:
UserAId
UserBId
NumberOfConnections
then using curos I iterate through each row in the table, then I make select statement:
if in user2user table there is record which has UserAId = UserAId from CDR record and UserBId = UserBId from CDR record then increase NumberOfConnections.
otherwise insert such a row which NumebrOfConnections = 1.
Quite simple task and it works as I said using cursor but it is very bad in performance (estimated time at my computer ~60 h).
I heard about Sql Server Integration Services that it has got better performance when we are talking about such big tables.
Problem is that I have no idea how to customize SSIS package for creating such task.
If anyone has got any idea how to help me, any good resources etc I would be really thankful.
Maybe there is any other good solution to make it work faster. I used indexes and variable tables and so on and performance is still pure.
thanks for help,
P.S.
This is script which I wrote and execution of this takes sth like 40 - 50 h.
DECLARE CDR_cursor CURSOR FOR
SELECT CDRId, SubscriberAId, BNumber
FROM dbo.CDR
OPEN CDR_cursor;
FETCH NEXT FROM CDR_cursor
INTO #CdrId, #SubscriberAId, #BNumber;
WHILE ##FETCH_STATUS = 0
BEGIN
--here I check if there is a user with this number (Cause in CDR i only have SubscriberAId --and BNumber so that I need to check which one user is this (I only have users from
--network so that each time I cant find this user I add one which is outide network)
SELECT #UserBId = (Select UserID from dbo.Number where Number = #BNumber)
IF (#UserBId is NULL)
BEGIN
INSERT INTO dbo.[User] (ID, Marked, InNetwork)
VALUES (#OutUserId, 0, 0);
INSERT into dbo.[Number](Number, UserId) values (#BNumber, #OutUserId);
INSERT INTO dbo.User2User
VALUES (#SubscriberAId, #OutUserId, 1)
SET #OutUserId = #OutUserId - 1;
END
else
BEGIN
UPDATE dbo.User2User
SET NumberOfConnections = NumberOfConnections + 1
WHERE User1ID = #SubscriberAId AND User2ID = #UserBId
-- Insert the row if the UPDATE statement failed.
if(##ROWCOUNT = 0)
BEGIN
INSERT INTO dbo.User2User
VALUES (#SubscriberAId, #UserBId, 1)
END
END
SET #Counter = #Counter + 1;
if((#Counter % 100000) = 0)
BEGIN
PRINT Cast (#Counter as NVarchar(12));
END
FETCH NEXT FROM CDR_cursor
INTO #CdrId, #SubscriberAId, #BNumber;
END
CLOSE CDR_cursor;
DEALLOCATE CDR_cursor;
The thing about SSIS is that it probably won't be much faster than a cursor. It's pretty much doing the same thing: reading the table record by record, processing the record and then moving to the next one. There are some advanced techniques in SSIS like sharding the data input that will help if you have heavy duty hardware, but without that it's going to be pretty slow.
A better solution would be to write an INSERT and an UPDATE statement that will give you what you want. With that you'll be better able to take advantage of indices on the database. They would look something like:
WITH SummaryCDR AS (UserAId, UserBId, Conns) AS
(
SELECT UserAId, UserBId, COUNT(1) FROM CDR
GROUP BY UserAId, UserBId)
UPDATE user2user
SET NumberOfConnections = NumberOfConnections + SummaryCDR.Conns
FROM SummaryCDR
WHERE SummaryCDR.UserAId = user2user.UserAId
AND SummaryCDR.UserBId = user2user.UserBId
INSERT INTO user2user (UserAId, UserBId, NumberOfConnections)
SELECT CDR.UserAId, CDR.UserBId, Count(1)
FROM CDR
LEFT OUTER JOIN user2user
ON user2user.UserAId = CDR.UserAId
AND user2user.UserBId = CDR.UserBId
WHERE user2user.UserAId IS NULL
GROUP BY CDR.UserAId, CDR.UserBId
(NB: I don't have time to test this code, you'll have to debug it yourself)
is this what you need?
select
UserAId, UserBId, count(CDRid) as count_connections
from cdr
group by UserAId, UserBId
Could you break the conditional update/insert into two separate statements and get rid of the cursor?
Do the INSERT for all the NULL rows and the UPDATE for all the NOT NULL rows.
Why are you even considering doing row-by-row processing on a table that size? You know you can use the merge statment and insert or update and it will be faster. Or you could write an update to insert all rows that need updating in one set-based stament and an insert to insert alll rows when the row doesn't exist in one set-based statement.
Stop using the values clause and use an insert with joins instead. Same thing with updates. If you need extra complexity the case stamenet will probably give you all you need.
In general stop thinking of row-by-row processing. If you can write a select for the cursor, you can write a set-based statement to do the work 99.9% of the time.
You may still want a cursor with a table this large but one to process batches of data (for instance a 1000 records at time) not one to run ro-by-row.