Better choice instead of functions for multiple inserts - mysql

in my sql database I have a procedure that needs to insert in a table tons of randomly generated records.
Something like this:
insert into table_AAA
SELECT
, round(X + rand() * 10 - rand() * 10 )
, round(Y + rand() * 10 - rand() * 10 )
FROM db_numbers d /*very big table containing just 1 column with numbers from 1 to 1M*/
limit 100000;
it takes 3 seconds, sound reasonable compared to other procedure of my servers.
Then I had to make that if the result of the round () is >=0
To do it I made a function:
CREATE FUNCTION `fn_normalize`(`p_value` INT) RETURNS int(11)
BEGIN
declare v_output INT;
IF p_value < 0 THEN
SET v_output = 0 ;
ELSE SET v_output = p_value ;
END IF;
RETURN (v_output);
END
and my insert became:
insert into table_AAA
SELECT
, fn_normalize(round(X + rand() * 10 - rand() * 10 ))
, fn_normalize(round(Y + rand() * 10 - rand() * 10 ))
FROM db_numbers d /*very big table containing just 1 column with numbers from 1 to 1M*/
limit 100000;
very slow, 10x the original one, probably because the function works on each value individually
I thought to use CASE WHEN:
insert into table_AAA
SELECT
, case when(round(X + rand() * 10 - rand() * 10 )) < 0 then 0 else (round(X + rand() * 10 - rand() * 10 )) end
, case when(round(Y + rand() * 10 - rand() * 10 )) < 0 then 0 else (round(X + rand() * 10 - rand() * 10 )) end
FROM db_numbers d /*very big table containing just 1 column with numbers from 1 to 1M*/
limit 100000;
But the else will re-run the rand function, so I cannot be sure is a positive number. Using rand(x) is not an option, because I need the most randomness of values.
Make an update after the insert is even worst.
Am I missing some obvious alternative?
Thanks a lot

YOu can use a subquery:
insert into table_AAA
SELECT CASE WHEN v.x<0 THEN 0 ELSE v.x END,
CASE WHEN v.y<0 THEN 0 ELSE v.y END,
FROM (
SELECT
round(X + rand() * 10 - rand() * 10 ) AS x
, round(Y + rand() * 10 - rand() * 10 ) AS y
FROM db_numbers d /*very big table containing just 1 column with numbers from 1 to 1M*/
limit 100000) v;

Much simpler (and faster) function body:
RETURN GREATEST(0, p_value);
Or, applying that to davide's solution:
insert into table_AAA
SELECT
GREATEST(0, round(X + rand() * 10 - rand() * 10 )) AS x,
GREATEST(0, round(Y + rand() * 10 - rand() * 10 )) AS y
FROM db_numbers d /* table of numbers from 1 to 1M*/
limit 100000) v;

Related

Is there any way to group and random some data in one sql?

I have a table with id from 1 to 10, now I need to random some data from 1 to 2, 3 to 5 and 6 to 10, such as random select 1,4,9, is there any way using one sql to resolve it?
SELECT tablename.*
FROM tablename
JOIN ( SELECT ROUND(1 * RAND() + 1) random UNION ALL
SELECT ROUND(2 * RAND() + 3) UNION ALL
SELECT ROUND(4 * RAND() + 6) ) randoms ON tablename.id = randoms.random

Relevancy MySQL query to replace stored procedure

I've taken over a project built by my predecessor. That project contains a stored procedure originally taken from here:
https://stackoverflow.com/a/9100182/439925
Currently this is preventing us from updating MySQL, so i have been attempting to remove it and replace the calls to subStringCount() with an adjustment to the top answer on that question.
(( Round (( Char_length(`title`) - Char_length(REPLACE(`title`, 'info', "")) ) / Char_length('info')) * 30 )) AS `title_score`,
The queries are used to count the number of times a search string occurs in a number of fields and then order by the total. Unfortunately i can't get the new query results to match the old one.
The 2 full queries are as follows:
Old WITH the stored Proc:
SELECT SQL_CALC_FOUND_ROWS `temp`.*,
( `title_score` + `source_score`
+ `abstract_score` + `authors_score`
+ `drugs_score` + `uploader_score`
+ `area_score`
+ Ifnull(`document_content_score`, 0) ) AS
`relevance`
FROM (SELECT `kb_uploads`.*,
(( Substringcount(`title`, 'info') * 30 )) AS `title_score`,
(( Substringcount(`source`, 'info') * 15 )) AS`source_score`,
(( Substringcount(`abstract`, 'info') * 20 ) AS`abstract_score`,
(( Substringcount(`authors`, 'info') * 30 )) AS `authors_score`,
(( Substringcount(`drugs`, 'info') * 20 )) AS `drugs_score`,
(( Substringcount(`kb_users`.`name`, 'info') * 20 )) AS `uploader_score`,
(( Substringcount(`kb_upload_areas`.`name`, 'info') * 20 )) AS `area_score`,
( `content_tbl`.`index_score` * 1 ) AS `document_content_score`
FROM `kb_uploads`
LEFT JOIN `kb_users`
ON `kb_users`.`id` = `kb_uploads`.`uploader`
LEFT JOIN `kb_upload_areas`
ON `kb_upload_areas`.`id` = `kb_uploads`.`area`
LEFT JOIN (SELECT `upload`,
Sum(`weighting`) AS `index_score`
FROM `kb_search_index`
WHERE `word` = 'info'
GROUP BY `upload`) AS `content_tbl`
ON `content_tbl`.`upload` = `kb_uploads`.`id`) AS `temp`
WHERE `is_deleted` = 0
HAVING `relevance` > 0
ORDER BY `relevance` DESC
LIMIT 10 OFFSET 0
New WITHOUT the stored Proc:
SELECT SQL_CALC_FOUND_ROWS `temp`.*,
( `title_score` + `source_score`
+ `abstract_score` + `authors_score`
+ `drugs_score` + `uploader_score`
+ `area_score`
+ Ifnull(`document_content_score`, 0) ) AS
`relevance`
FROM (SELECT `kb_uploads`.*,
(( Round (( Char_length(`title`) - Char_length(REPLACE(`title`, 'info', "")) ) / Char_length('info')) * 30 )) AS `title_score`,
(( Round (( Char_length(`source`) - Char_length(REPLACE(`source`,'info' , "")) ) / Char_length('info')) * 15 )) AS `source_score`,
(( Round (( Char_length(`abstract`) - Char_length(REPLACE(`abstract`, 'info', "" ) ) ) / Char_length('info')) * 20 )) AS `abstract_score`,
(( Round (( Char_length(`authors`) - Char_length(REPLACE(`authors`,'info', "")))/Char_length('info')) * 30 )) AS `authors_score`,
(( Round (( Char_length(`drugs`) - Char_length(REPLACE(`drugs`,'info',"")) ) / Char_length('info')) * 20 )) AS `drugs_score`,
(( Round (( Char_length(`kb_users`.`name`) - Char_length(REPLACE(`kb_users`.`name`,'info',""))) / Char_length('info')) * 20 )) AS `uploader_score`,
(( Round (( Char_length(`kb_upload_areas`.`name`) - Char_length(REPLACE(`kb_upload_areas`.`name`,'info', "")) ) / Char_length('info'))* 20 )) AS`area_score`,
( `content_tbl`.`index_score` * 1 ) AS `document_content_score`
FROM `kb_uploads`
LEFT JOIN `kb_users`
ON `kb_users`.`id` = `kb_uploads`.`uploader`
LEFT JOIN `kb_upload_areas`
ON `kb_upload_areas`.`id` = `kb_uploads`.`area`
LEFT JOIN (SELECT `upload`,
Sum(`weighting`) AS `index_score`
FROM `kb_search_index`
WHERE `word` = 'info'
GROUP BY `upload`) AS `content_tbl`
ON `content_tbl`.`upload` = `kb_uploads`.`id`) AS `temp`
WHERE `is_deleted` = 0
HAVING `relevance` > 0
ORDER BY `relevance` DESC
LIMIT 10 OFFSET 0
There are 2 main issues.
1) Ifnull is not working in the NEW query. The table column contains mostly null instead of 0
2) The relevancy numbers in the new query don't match the numbers in the old one, possibly something to do with IFNULL not working.
The full queries are constructed in PHP, i have left the code out as the logic hasnt changed, only the string concats to replace the Stored Proc.

How to use value from two different rows of a table in another table

I have a MySQL table with the following structure and data:
Increments
id emp_id starting_salary increment_rate increment_frequency
2 340 5000 250 1
3 340 5000 250 4
I need to have aliases, a and b which will hold some value based on the following formula:
starting_salary + (increment_rate * increment_frequency)
To be precise, I want a = 5250 (based on a = (5000 + (250 * 1))) and b = 6000 (based on b = (5000 + (250 * 4)))
Now I have another table with the following data:
PaySlips
id employee_id salary_month arrear
173824 340 '2015-06-01' 2350
I want to join a and b that I got from the table Increments with table PaySlips. And I want to use a and b in the following way:
((a * 8) / 30 + (b * 22) / 30)
My alias will be basic_salary. So basic_salary will hold this value from the above calculation:
basic_salary = ((a * 8) / 30 + (b * 22) / 30)
= ((5250 * 8) / 30 + (6000 *22) / 30)
= (1400 + 4400)
= 5800
I've got no idea how to do this. Can anyone please help me?
All I got are the common columns in both tables - emp_id and employee_id and I can join both tables using these columns. I just can't figure out how I can store the above values and organize the calculation inside my query.
Sample query:
SELECT x.id, x.employee_id,
(*my calculation using a and b from table Increments*) AS basic_salary,
x.salary_month, x.arrear
FROM PaySlips x
JOIN Increments y
ON x.employee_id = y.emp_id
For determining a:
SELECT
(
starting_salary +
(increment_rate * increment_frequency)
) AS a
FROM Increments
WHERE id = 2
And for determining b:
SELECT
(
starting_salary +
(increment_rate * increment_frequency)
) AS b
FROM Increments
WHERE id = 3
MySQL version: 5.2
I'm not clear on all the details, for example what should happen if there are three rows for one employee in increments? Anyhow, here's a sketch to start with:
select employee_id
, ((a * 8) / 30 + (b * 22) / 30) as basic_salary
from (
select x.employee_id
, min(starting_salary + (increment_rate * increment_frequency)) as a
, max(starting_salary + (increment_rate * increment_frequency)) as b
, x.salary_month, x.arrear
from payslips x
join increments y
on x.employee_id = y.emp_id
group by x.employee_id, x.salary_month, x.arrear
) as t
If id 2 and 3 are the criteria (I assumed they are examples) you can use a case statement like:
select employee_id
, ((a * 8) / 30 + (b * 22) / 30) as basic_salary
from (
select x.employee_id
, max(starting_salary + (increment_rate * case when y.id = 2 then increment_frequency end )) as a
, max(starting_salary + (increment_rate * case when y.id = 3 then increment_frequency end)) as b
, x.salary_month
, x.arrear
from payslips x
join increments y
on x.employee_id = y.emp_id
group by x.employee_id, x.salary_month, x.arrear
) as t;
In this case it does not matter what aggregate you use, it is to get rid of the row that contains null.
based on the requirements you added i think something like this will solve your problems:
SELECT PS.id, PS.employee_id, ((A.value * 8) / 30 + (B.value * 22) / 30) AS basic_salary
FROM PaySlips AS PS
JOIN (
SELECT I.emp_id, I.starting_salary + (increment_rate * increment_frequency) AS VALUE
FROM Increments AS I
WHERE I.id = 2
) AS A
ON A.emp_id = PS.employee_id
JOIN (
SELECT I.emp_id, I.starting_salary + (increment_rate * increment_frequency) AS value
FROM Increments AS I
WHERE I.id = 3
) AS B
ON B.emp_id = PS.employee_id
This version might need some alteration if it's not working on your real scenario, but please feel free to tell if anything else needs amending.
Hope it helps.
For determining and setting #a variable:
SET #a := (SELECT
(
starting_salary +
(increment_rate * increment_frequency)
) AS a
FROM Increments
WHERE id = 2);
And for determining and setting #b variable:
SET #b := (SELECT
(
starting_salary +
(increment_rate * increment_frequency)
) AS b
FROM Increments
WHERE id = 3);
Then you can use #a and #b in your main query;
you can test simply by
SELECT #a as a;
SELECT #b as b;
SELECT
x.id,
x.employee_id,
(y.a * 8) / 30 + (y.b * 22) / 30 as basic_salary,
x.salary_month,
x.arrear
FROM PaySlips x
JOIN (
select t1.emp_id, t1.a, t2.b
from (
select
emp_id,
starting_salary + increment_rate * increment_frequency as a
from Increments
where id = 2
) as t1
join (
select
emp_id,
starting_salary + increment_rate * increment_frequency as b
from Increments
where id = 3
) as t2
on t1.emp_id = t2.emp_id
) as y
ON x.employee_id = y.emp_id

MYSQL join/union with conditional limit

In this query I want a total of x number of records returned. within that query I have several sub-queries where I can't be sure if they'll return the max number of records. if one result is less than it's max limit I want to populate the remaining slots with the next query and so on. I can't do math inside a limit clause so I'm still trying to figure out how to do it. here is what I would do if math was available inside the limit clause.
select *
from
(
(select * from profile where size='local' order by rand() limit 7) as local
join
(select * from profile where size='regional' order by rand() limit (13-count(local.id)) as regional
join
(select * from profile where size='national' order by rand() limit (19-(count(local.id)+count(regional.id))) as national
join
(select * from profile where size='international' order by rand() limit (25-(count(local.id)+count(regional.id)+count(national.id)))) as international
)
I might have done this a needlessly complicated way, but it does seem to work:-
SELECT id, size
FROM
(
SELECT id, size,
#SeqLocal:=IF(size="local", IF(#SeqLocal <= 7, #SeqLocal + 1, #SeqLocal), #SeqLocal) AS SeqLocal,
#SeqRegional:=IF(size="regional", IF(#SeqLocal + #SeqRegional <= 14, #SeqRegional + 1, #SeqRegional), #SeqRegional) AS SeqRegional,
#SeqNational:=IF(size="national", IF(#SeqLocal + #SeqRegional + #SeqNational <= 21 , #SeqNational + 1, #SeqNational), #SeqNational) AS SeqNational,
#SeqInternational:=IF(size="international", IF(#SeqLocal + #SeqRegional + #SeqNational + #SeqInternational <= 28, #SeqInternational + 1, #SeqInternational), #SeqInternational) AS SeqInternational
FROM
(
select *
from profile
where size IN ("local", "regional", "national", "international")
order by FIELD(size, "local", "regional", "national", "international"), rand()
) Sub1
CROSS JOIN (SELECT #SeqLocal:=0, #SeqRegional:=0, #SeqNational:=0, #SeqInternational:=0) Sub2
) Sub3
WHERE (size = "local" AND SeqLocal != #SeqLocal)
OR (size = "regional" AND SeqRegional != #SeqRegional)
OR (size = "national" AND SeqNational != #SeqNational )
OR (size = "international" AND SeqInternational != #SeqInternational)
Sqlfiddle here:-
http://www.sqlfiddle.com/#!2/cc3b884/14

How to get mysql random integer range?

I am trying to generate a random integer for each row I select between 1 and 60 as timer.
SELECT downloads.date, products.*, (FLOOR(1 + RAND() * 60)) AS timer
I have searched and keep coming up to this FLOOR function as how to select a random integer in a range. This is giving me a 1 for every row.
What am I missing?
I am on mysql 5.0.75
Heres the rest of the query I belive it might be a nesting issue
SELECT *
FROM (
SELECT downloads.date, products.*, FLOOR(1 + (RAND() * 60)) AS randomtimer,
(
SELECT COUNT( * )
FROM distros
WHERE distros.product_id = products.product_id
) AS distro_count,
(SELECT COUNT(*) FROM downloads WHERE downloads.product_id = products.product_id) AS true_downloads
FROM downloads
INNER JOIN products ON downloads.product_id = downloads.product_id
) AS count_table
WHERE count_table.distro_count > 0
AND count_table.active = 1
ORDER BY count_table.randomtimer , count_table.date DESC LIMIT 10
This is working for me. Your mysql version maybe?
SELECT id, (FLOOR( 1 + RAND( ) *60 )) AS timer
FROM users
LIMIT 0 , 30
The output of the RAND function will always be a value between 0 and 1.
Try this:
SELECT downloads.date, products.*, (CAST(RAND() * 60 AS UNSIGNED) + 1) AS timer
Old question, but always actual problem.
Here a way to create a MySQL function random_integer() based on manual :
CREATE FUNCTION random_integer(value_minimum INT, value_maximum INT)
RETURNS INT
COMMENT 'Gets a random integer between value_minimum and value_maximum, bounds included'
RETURN FLOOR(value_minimum + RAND() * (value_maximum - value_minimum + 1));
SELECT ALL random_integer(1, 60) AS timer;
I'm running your query and it does give me a random number for each row.... maybe has something to do with the name of the random (timer)?
You can increase the number multiplied by the number of records in the table.
SELECT id,
(FLOOR( (SELECT MIN(id) FROM your_table ) + RAND( ) * 1000000 ) ) AS timer
FROM your_table
LIMIT 0 , 30