Mysql, select many rows and assign different values to each row transactionally - mysql

I have a simple MySQL 8 table like this:
CREATE TABLE IF NOT EXISTS `test_codes` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`segment_id` int(11) unsigned NOT NULL,
`code_id` int(11) DEFAULT 0,
PRIMARY KEY (`id`)
);
The table is populated with some random values
INSERT INTO `test_codes` (`segment_id`) VALUES ('1');
INSERT INTO `test_codes` (`segment_id`) VALUES ('1');
INSERT INTO `test_codes` (`segment_id`) VALUES ('1');
INSERT INTO `test_codes` (`segment_id`) VALUES ('1');
INSERT INTO `test_codes` (`segment_id`) VALUES ('1');
INSERT INTO `test_codes` (`segment_id`) VALUES ('1');
INSERT INTO `test_codes` (`segment_id`) VALUES ('2');
INSERT INTO `test_codes` (`segment_id`) VALUES ('2');
INSERT INTO `test_codes` (`segment_id`) VALUES ('2');
INSERT INTO `test_codes` (`segment_id`) VALUES ('1');
INSERT INTO `test_codes` (`segment_id`) VALUES ('1');
My applications has the following task: A request comes with an array of codes eg [1,200,10,18] and I need to get 4 rows (equal to array size) from the database where the code = 0 and update the code_id at each row with the values 1,200, 10, 18 transitionally.
A concurrent request that wants to update the code from another running thread, should not access the selected rows of the first thread.
How can I do this?
After the update the first selected row will have code_id 1, the second selected row code_id 200, the third 10 and the last one 18. In other words the task must find rows with unassigned codes (code_id=0) and set a value to each row.
Link: http://sqlfiddle.com/#!9/60e555/1

You can do it with a single statement which you may build in the app
update test_codes
join
(
select id, row_number() over (order by id) rn
from test_codes
where code_id = 0
) t on t.id = test_codes.id
join (
-- a table of new values with their positions
select 1 rn, 1 val union all
select 2, 200 union all
select 3, 10 union all
select 4, 18
) v on v.rn = t.rn
set code_id = v.val
db<>fiddle

MySQL doesn't support arrays. I would suggest that you first load the array into a table. This is just a convenience, but it is handy.
Then you can use a complex update with join to handle this:
update test_codes tc join
(select tc2.*,
row_number() over (order by rand()) as seqnum
from test_codes tc2
) tc2
on tc2.id = tc.id join
(select nc.*,
row_number() over (order by code_id) as seqnum
from new_codes nc
) nc
on tc2.seqnum = nc.seqnum
set tc.code_id = nc.code_id;
EDIT:
You can construct the query directly from the codes:
update test_codes tc join
(select tc2.*,
row_number() over (order by rand()) as seqnum
from test_codes tc2
) tc2
on tc2.id = tc.id join
(select ? as code, 1 as seqnum union all
select ? as code, 2 union all
. . .
) nc
on tc2.seqnum = nc.seqnum
set tc.code_id = nc.code_id;

Related

Select sum with input from grouped query

I need to make a query that selects a grouped collection of rows from a table based on user input conditions, and then in the select i will sum data from a subset of the rows.
The setup is rather expansive to describe in a post, so here is a demostration of the problem in the simplest way i can make it:
We have this table: DemoTable
ID
StaticKey
GroupKey
Value
1
A
A
2
2
A
A
2
3
A
B
2
4
A
B
2
5
A
C
2
6
A
C
2
I make a select and groups on "StaticKey".
What i would then like to do, is to, in the select clause, to select the sum of a subset of the values from the groupped data:
select
DT.GroupKey,
(select sum(D.Value) from DemoTable D where D.ID in (DT.ID) and D.GroupKey = 'A') as 'Sum of A''s',
(select COUNT(D.ID) from DemoTable D where D.ID in (DT.ID) and D.GroupKey = 'A') as 'Count of A''s'
from DemoTable DT
group by DT.StaticKey;
I hoped that the sum would result in a sum of 4 and a count of 2, but i get 2 and 1. So the input to the "select sum" seems to be just one id and not the collected ids.
GroupKey
Sum of A's
Count of A's
A
2
1
If i add a group_concat of DT.ID i get them comma separated - but is it posible to get them as a collection i can use as input to the selects?
Heres sql to create the table and queries:
CREATE TABLE DemoTable
(
ID INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
GroupKey varchar(200) null default null,
StaticKey varchar(200) not null default 'A',
Value varchar(200) null default null,
PRIMARY KEY (ID)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8;
insert into DemoTable (GroupKey, Value) values ('A', 2);
insert into DemoTable (GroupKey, Value) values ('A', 2);
insert into DemoTable (GroupKey, Value) values ('B', 2);
insert into DemoTable (GroupKey, Value) values ('B', 2);
insert into DemoTable (GroupKey, Value) values ('C', 2);
insert into DemoTable (GroupKey, Value) values ('C', 2);
select DT.GroupKey,
(select sum(D.Value) from DemoTable D where D.ID in (DT.ID) and D.GroupKey = 'A') as 'Sum of A''s',
(select COUNT(D.ID) from DemoTable D where D.ID in (DT.ID) and D.GroupKey = 'A') as 'Count of A''s'
from DemoTable DT
group by DT.StaticKey;
DROP TABLE DemoTable;
More simple:
select GroupKey,
sum(Value) as sum_of_A,
sum(GroupKey='A') as count_of_A
from DemoTable
where GroupKey='A'
group by GroupKey;
https://dbfiddle.uk/sdYlTw57
Isn't it always D.ID in (DT.ID)?
select
DT.GroupKey,
(select sum(D.Value) from DemoTable D where D.GroupKey = 'A') as 'Sum of A''s',
(select COUNT(D.ID) from DemoTable D where D.GroupKey = 'A') as 'Count of A''s'
from DemoTable DT
group by DT.StaticKey;
It does the job but perhaps it's too simple...
You can also try this.
SELECT DT.GroupKey,
SUM(DT.Value) AS 'Sum of A''s',
COUNT(DT.ID) AS 'Count of A''s'
FROM DemoTable DT
WHERE DT.GroupKey = 'A'
GROUP BY DT.StaticKey;

MySQL: ORDER BY and GROUP BY together

I recently upgraded to MySQL 5.7.22 and my query stopped working. I have two tables "items" and "packages" where I'm trying to output a row for each item including a column for the package with the minimum price per unit, but ignore packages that have a price per unit set to 0.
Here's a minimal sample of tables and data:
CREATE TABLE `items` (
`id` int(11) NOT NULL
);
CREATE TABLE `packages` (
`item_id` int(11) NOT NULL,
`price_per_unit` float(16,6) DEFAULT 0
);
INSERT INTO `items` (`id`) VALUES
(1),
(2),
(3);
INSERT INTO `packages` (`item_id`, `price_per_unit`) VALUES
(1, 0.45),
(1, 0),
(1, 0.56),
(1, 0.34);
Here's the query:
SELECT
*
FROM
(
SELECT
items.id,
NULLIF(pkgs.ppu, 0) AS mppu
FROM
items
LEFT JOIN
(
SELECT
item_id,
price_per_unit AS ppu
FROM
packages
) AS pkgs ON pkgs.item_id = items.id
ORDER BY
IFNULL(mppu, 9999)
) X
GROUP BY
X.id
I was setting the zero values to null and then bumping their values to be much higher during the ordering. There must be a better way (especially since this method doesn't work any longer).
The expected output for this data is:
id mppu
1 0.34
2 null
3 null
I think your query is a bit too complex. What about this?
SELECT i.id,IFNULL(Min(p.price_per_unit), 'NONE')
FROM items i
LEFT JOIN packages p
ON ( i.id = p.item_id )
WHERE p.price_per_unit > 0
OR p.price_per_unit IS NULL
GROUP BY i.id
See this fiddle. I used this data:
INSERT INTO `items` (`id`) VALUES
(1),(2),(3);
INSERT INTO `packages` (`item_id`, `price_per_unit`) VALUES
(1, 0.45),
(1, 0),
(1, 0.56),
(1, 0.34),
(2, 9.45),
(2, 0),
(2, 0.56),
(2, 0.14);
And got this result:
id IFNULL(min(p.price_per_unit),'None')
1 0.340000
2 0.140000
3 None
Agree with GL,
SELECT * FROM GROUP BY
is not predictable .
i will rewrite the query with :
SELECT a.*,b.min_price_per_unit
FROM items a
LEFT JOIN (
SELECT item_id
,min(CASE
WHEN price_per_unit = 0
THEN 9999
ELSE price_per_unit
END) AS min_price_per_unit
FROM packages
GROUP BY item_id
) b ON a.id = b.item_id;

get rows from a table where value of field x is maximum

I have two tables myTable and myTable2 in a mysql database:
CREATE TABLE myTable (
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
number INT,
version INT,
date DATE
) ENGINE MyISAM;
INSERT INTO myTable
(`id`, `number`, `version`, `date`)
VALUES
(1, '123', '1', '2016-01-12'),
(2, '123', '2', '2016-01-13'),
(3, '124', '1', '2016-01-14'),
(4, '124', '2', '2016-01-15'),
(5, '124', '3', '2016-01-16'),
(6, '125', '1', '2016-01-17')
;
CREATE TABLE myTable2 (
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
myTable_id INT
) ENGINE MyISAM;
INSERT INTO myTable2
(`id`, `myTable_id`)
VALUES
(1, 1),
(2, 1),
(3, 2),
(4, 2),
(5, 3),
(6, 3),
(7, 4),
(8, 4),
(9, 4),
(10, 5),
(11, 6)
;
The field myTable2.myTable_id is a foreign key of myTable.Id.
I would like to get all the rows from myTable where myTable2.myTable_id = myTable.Id and the value of the field version in myTable is the maximum for every corresponding value for the field number in myTable.
I tried something like this:
SELECT
*
FROM
myTable,
myTable2
WHERE
myTable.version = (SELECT MAX(myTable.version) FROM myTable)
But the above query does not return the correct data. The correct query should output this:
Id number version date
2 123 2 2016-01-13
5 124 3 2016-01-16
6 125 1 2016-01-17
Please help!
One way to do this is to get the max version for each number in myTable in a derived table and join with that:
SELECT DISTINCT
m.*
FROM
myTable m
JOIN
myTable2 m2 ON m.id = m2.myTable_id
JOIN
(
SELECT number, MAX(version) AS max_version
FROM myTable
GROUP BY number
) AS derived_table
ON m.number = derived_table.number
AND m.version = derived_table.max_version
With your sample data this produces a result like this:
id number version date
6 125 1 2016-01-17
5 124 3 2016-01-16
2 123 2 2016-01-13
your Query is logically wrong. Here is the correct one
SELECT
*
FROM
myTable,
myTable2
WHERE
(myTable.version,myTable.number) in
(SELECT MAX(myTable.version),number FROM myTable group by number)
and myTable.id=myTable2.id
Here is the sqlfiddle http://sqlfiddle.com/#!9/74a67/4/0
This is the query posted for the previous edited question
SELECT * FROM myTable
inner join myTable2 on myTable.id = myTable2.mytable_id
WHERE (version, number) in
(SELECT MAX(version), number FROM myTable group by number)
Try this solution with using subquery simply as:
# Selecting desired result..
SELECT t1.id, t1.number, t1.version, t1.date
FROM myTable As t1 JOIN
# subquery to select max version and its corresponding
# number form myTable
(SELECT number, max(version) As max_ver FROM myTable
GROUP BY number
) As t2 ON t1.number = t2.number and t1.version = t2.max_ver
# Now checking for foreign key..
WHERE t1.id IN (SELECT mytable_id FROM myTable2);
Was it helpful..

How to optmize query with a query with multiple GROUP BY's,sub queries and WHERE IN over a large table?

I am working on a scraping project to crawl items and their scores over different schedules.Schedule is a user defined period (date) when the script is intended to run.
Table structure is as follows:
--
-- Table structure for table `test_join`
--
CREATE TABLE IF NOT EXISTS `test_join` (
`schedule_id` int(11) NOT NULL,
`player_name` varchar(50) NOT NULL,
`type` enum('celebrity','sportsperson') NOT NULL,
`score` int(11) NOT NULL,
PRIMARY KEY (`schedule_id`,`player_name`,`type`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
--
-- Dumping data for table `test_join`
--
INSERT INTO `test_join` (`schedule_id`, `player_name`, `type`, `score`) VALUES
(1, 'sachin', 'sportsperson', 100),
(1, 'ganguly', 'sportsperson', 80),
(1, 'dravid', 'sportsperson', 60),
(1, 'sachin', 'celebrity', 100),
(2, 'sachin', 'sportsperson', 120),
(2, 'ganguly', 'sportsperson', 100),
(2, 'sachin', 'celebrity', 120);
The scraping is done over periods and for each schedule it is expected to have about 10k+ entries.The schedules could be made in daily basis,hence the data would grow to be be around 2 million in 5-6 months.
Over this data I need to perform queries to aggregate the player who come across each schedules in a selected range of schedules.
For example:
I need aggregate same players who come across multiple schedules. If schedule 1 and 2 are selected,items which come under both of the schedules only will be selected.
I am using the following query to aggregate results based on the type,
For schedule 1:
SELECT fullt.type,COUNT(*) as count,SUM(fullt.score) FROM
(SELECT tj.*
FROM `test_join` tj
RIGHT JOIN
(SELECT `player_name`,`type`,COUNT(`schedule_id`) as c FROM `test_join` WHERE `schedule_id` IN (1,2) GROUP BY `player_name`,`type` HAVING c=2) stj
on tj.player_name = stj.player_name
WHERE tj.`schedule_id`=1
GROUP BY tj.`type`,tj.`player_name`)AS fullt
GROUP BY fullt.type
Reason for c = 2;
WHERE `schedule_id` IN (1,2) GROUP BY `player_name`,`type` HAVING c=2
Here we are selecting two schedules,1 and 2.Hence the count 2 is taken to make the query to to fetch records which belongs to both the schedules and occurs twice.
It would generate a results as follows,
Schedule 1 :Expected Results
Schedule 2 :Expected Results
This is my expected result and the query returns the results as above.
(In the real case I have to work across pretty big MySQL tables)
On my understanding of standardized MySQL queries, using sub queries,WHERE IN, varchar comparison fields ,multiple GROUP BY's would affect in the query performance.
I need the aggregate results in real time and query speed and well as standards are a concern too.How this could be optimized for better performance in this context.
EDIT:
I had reduced sub queries now:
SELECT fullt.type,COUNT(*) as count,SUM(fullt.score) FROM (
SELECT t.*
FROM `test_join` t
INNER JOIN test_join t1 ON t.`player_name` = t1.player_name AND t1.schedule_id = 1
INNER JOIN test_join t2 ON t.player_name = t2.player_name AND t2.schedule_id = 2
WHERE t.schedule_id = 2
GROUP BY t.`player_name`,t.`type`) AS fullt
GROUP BY fullt.type
Is this a better way to do so.I had replaced WHERE IN with JOINS.
Any advise would be highly appreciated.I would be happy to provide any supporting information if needed.
try below SQL Query in MYSQL:
SELECT tj.`type`,COUNT(*) as count,SUM(tj.`score`) FROM
`test_join` tj
where tj.`schedule_id`=1
and `player_name` in
(
select tj1.`player_name` from `test_join` tj1
group by tj1.`player_name` having count(tj1.`player_name`) > 1
)
group by tj.`type`
Actuallly I tried same data in Sybase as i dont have MySQL installed in my machine.It worked as exepected !
CREATE TABLE #test_join
(
schedule_id int NOT NULL,
player_name varchar(50) NOT NULL,
type1 varchar(15) NOT NULL,
score int NOT NULL,
)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES
(1, 'sachin', 'sportsperson', 100)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(1, 'ganguly', 'sportsperson', 80)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(1, 'dravid', 'sportsperson', 60)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(1, 'sachin', 'celebrity', 100)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(2, 'sachin', 'sportsperson', 120)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(2, 'ganguly', 'sportsperson', 100)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(2, 'sachin', 'celebrity', 120)
select * from #test_join
Print 'Solution #1 : Inner join'
select type1,count(*),sum(score) from
#test_join
where schedule_id=1 and player_name in (select player_name from #test_join t1 group by player_name having count(player_name) > 1 )
group by type1
select player_name,type1,sum(score) Score into #test_join_temp
from #test_join
group by player_name,type1
having count(player_name) > 1
Print 'Solution #2 using Temp Table'
--select * from #test_join_temp
select type1,count(*),sum(score) from
#test_join
where schedule_id=1 and player_name in (select player_name from #test_join_temp )
group by type1
I hope This Helps :)

How to combine sql request with data in another table

I have two tables, first "users_counts"
id int(11) AUTO_INCREMENT
name varchar(250)
And I have second table "counts_data"
id int(11) AUTO_INCREMENT
id_user int(11)
count int(11)
date datetime
I want to select all records from the first table and get some data from a second, and then I want to merge they. I want create temp (for one request) column where collect last count with order by date in second table and second column where collect collect penultimate count with order by date in second table.
INSERT INTO `users_counts` (`id`,`name`) VALUES ('1','John');
INSERT INTO `users_counts` (`id`,`name`) VALUES ('2','Michael');
INSERT INTO `users_counts` (`id`,`name`) VALUES ('3','Den');
INSERT INTO `counts_data` (`id`,`id_user`, `count`, `date`) VALUES ('1','1', '200', '2012.09.09');
INSERT INTO `counts_data` (`id`,`id_user`, `count`, `date`) VALUES ('2','1', '212', '2012.09.01');
INSERT INTO `counts_data` (`id`,`id_user`, `count`, `date`) VALUES ('3','2', '20', '2012.01.09');
INSERT INTO `counts_data` (`id`,`id_user`, `count`, `date`) VALUES ('4','3', '210', '2012.02.09');
INSERT INTO `counts_data` (`id`,`id_user`, `count`, `date`) VALUES ('5','3', '2033', '2012.03.09');
INSERT INTO `counts_data` (`id`,`id_user`, `count`, `date`) VALUES ('6','3', '1', '2012.04.09');
In the end, after a request I want to get something like this
id name count count_before
1 John 200 212
2 Michael 20 0
3 Den 1 2033
Thank.
Another possible way to do this:
select uc.id,
uc.name,
(select count
from counts_data cd
where cd.id_user = uc.id
order by date desc limit 1) as count,
ifnull((select count
from counts_data cd
where cd.id_user = uc.id
order by date desc limit 1 offset 1),0) as count_before
from users_counts uc;
Since you only need one value from the counts_data for each row/record, you can use in-line queries in mySQL
SQL Fiddle
select uc.id
, uc.name
, cd1.count
, cd3.count as count_before
from users_counts uc
left join
counts_data cd1
on cd1.id_user = uc.id
and cd.date =
(
select max(date)
from counts_data cd2
where cd2.id_user = uc.id_user
)
left join
counts_data cd3
on cd3.id_user = uc.id
and cd.date =
(
select max(date)
from counts_data cd4
where cd4.id_user = uc.id_user
and cd4.date <> cd1.date
)