I am trying to fill db pre populated data from sql script where I have two type of constants or enums.
Platform: DG, NK
Department : KK, TG, LO, NP, UI, BG, ED, CC.
Task: To generate a sequential number using procedural loop and for each combination using above value we need to generate key and put in data base with count or sequence value.
Database columns are based on JPA abstract entity:
id, created_by, created_at, status, updated_by, updated_at, uuid, count, category_key
Now single row would be one combination which is formed using this pattern,
Department_Platform_SequenceNumber :: example => KK_DG_1,....KK_DG_10000, KK_NK_1,....KK_NK_10000
This is for 10k entries for 10k sequence of each combinations. It follows for other as well.
Approach:
DROP PROCEDURE KeyGeneration;
DELIMITER $$
CREATE PROCEDURE LoopDemo()
BEGIN
DECLARE x INT;
DECLARE dep VARCHAR(10);
DECLARE plat VARCHAR(10);
DECLARE str VARCHAR(30);
SET x = 1;
SET dep = 'KK'; # help to initialize enums or constants
SET plat = 'DG'; # help to initialize enums or constants
count_val: LOOP
IF x=10000 THEN
LEAVE count_val;
END IF;
SET str = CONCAT(dep,'_',plat,'_',x);
insert into counter_key values(id, created_by, created_at, status, updated_by, updated_at, uuid, x, str);
SET x = x + 1;
END LOOP;
END$$
DELIMITER ;
call LoopDemo();
but this is wrong since what I want is that atleast id to be updated if possible created_at and other fields as well also the loop will return last value I guess I want to get each value.
TABLE COLUMNS UPDATED
UPDATED: id, count, counter_key, status
#Akina answer applied as per my new table but syntax error
WITH RECURSIVE
number AS ( SELECT 1 number
UNION ALL
SELECT number + 1 FROM cte WHERE number < 10000 ),
platform AS ( SELECT 'DG' platform
UNION ALL
SELECT 'NK' ),
department AS ( SELECT 'KK' department
UNION ALL
SELECT 'TG'
UNION ALL
SELECT 'LO'
UNION ALL
SELECT 'NP'
UNION ALL
SELECT 'UI'
UNION ALL
SELECT 'BG'
UNION ALL
SELECT 'ED'
UNION ALL
SELECT 'CC' )
INSERT INTO counter_key
SELECT null, number, CONCAT_WS('_', department, platform, number), 1
FROM department
CROSS JOIN platform
CROSS JOIN number;
WITH RECURSIVE
number AS ( SELECT 1 number
UNION ALL
SELECT number + 1 FROM cte WHERE number < 10000 ),
platform AS ( SELECT 'DG' platform
UNION ALL
SELECT 'NK' ),
department AS ( SELECT 'KK' department
UNION ALL
SELECT 'TG'
UNION ALL
SELECT 'LO'
UNION ALL
SELECT 'NP'
UNION ALL
SELECT 'UI'
UNION ALL
SELECT 'BG'
UNION ALL
SELECT 'ED'
UNION ALL
SELECT 'CC' )
INSERT INTO counter_key
SELECT #id, #created_by, #created_at, #status, #updated_by, #updated_at, #uuid, #x,
CONCAT_WS('_', department, platform, number)
FROM department
CROSS JOIN platform
CROSS JOIN number;
where #id, #created_by, #created_at, #status, #updated_by, #updated_at, #uuid, #x must be replaced with literal values taken from somewhere.
Related
I need to separate values and store them in different variables in SQL,
for example
a='3100,3101,3102,....'
And the output should be
x=3100
y=3101
z=3102
.
.
.
create function [dbo].[udf_splitstring] (#tokens varchar(max),
#delimiter varchar(5))
returns #split table (
token varchar(200) not null )
as
begin
declare #list xml
select #list = cast('<a>'
+ replace(#tokens, #delimiter, '</a><a>')
+ '</a>' as xml)
insert into #split
(token)
select ltrim(t.value('.', 'varchar(200)')) as data
from #list.nodes('/a') as x(t)
return
end
GO
declare #cad varchar(100)='3100,3101,3102'
select *,ROW_NUMBER() over (order by token ) as rn from udf_splitstring(#cad,',')
token rn
3100 1
3101 2
3102 3
The results of the Parse TVF can easily be incorporated into a JOIN, or an IN
Declare #a varchar(max)='3100,3101,3102'
Select * from [dbo].[udf-Str-Parse](#a,',')
Returns
RetSeq RetVal
1 3100
2 3101
3 3102
The UDF if needed (much faster than recursive, loops, and xml)
CREATE FUNCTION [dbo].[udf-Str-Parse] (#String varchar(max),#Delimiter varchar(25))
Returns Table
As
Return (
with cte1(N) As (Select 1 From (Values(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)) N(N)),
cte2(N) As (Select Top (IsNull(DataLength(#String),0)) Row_Number() over (Order By (Select NULL)) From (Select N=1 From cte1 a,cte1 b,cte1 c,cte1 d) A ),
cte3(N) As (Select 1 Union All Select t.N+DataLength(#Delimiter) From cte2 t Where Substring(#String,t.N,DataLength(#Delimiter)) = #Delimiter),
cte4(N,L) As (Select S.N,IsNull(NullIf(CharIndex(#Delimiter,#String,s.N),0)-S.N,8000) From cte3 S)
Select RetSeq = Row_Number() over (Order By A.N)
,RetVal = LTrim(RTrim(Substring(#String, A.N, A.L)))
From cte4 A
);
--Orginal Source http://www.sqlservercentral.com/articles/Tally+Table/72993/
--Much faster than str-Parse, but limited to 8K
--Select * from [dbo].[udf-Str-Parse-8K]('Dog,Cat,House,Car',',')
--Select * from [dbo].[udf-Str-Parse-8K]('John||Cappelletti||was||here','||')
I suggest you to use following query, it's much faster than other functions like cross apply and udf.
SELECT
Variables
,S_DATA
FROM (
SELECT
Variables
,CASE WHEN LEN(LIST2)>0 THEN LTRIM(RTRIM(SUBSTRING(LIST2, NUMBER+1, CHARINDEX(',', LIST2, NUMBER+1)-NUMBER - 1)))
ELSE NULL
END AS S_DATA
,NUMBER
FROM(
SELECT Variables
,','+COMMA_SEPARETED_COLUMN+',' LIST2
FROM Tb1
)DT
LEFT OUTER JOIN TB N ON (N.NUMBER < LEN(DT.LIST2)) OR (N.NUMBER=1 AND DT.LIST2 IS NULL)
WHERE SUBSTRING(LIST2, NUMBER, 1) = ',' OR LIST2 IS NULL
) DT2
WHERE S_DATA<>''
and also you should create a table 'NUMBER' before running the above query.
CREATE TABLE TB (Number INT)
DECLARE #I INT=0
WHILE #I<1000
BEGIN
INSERT INTO TB VALUES (#I)
SET #I=#I+1
END
I need to find the most occurrences in a 10yr age range that can be Age 2 to 22, 15 to 25, 10 to 20, etc. in a table with name & age
I've created the SQL that returns the average age:
SELECT age, count(age)
FROM member
GROUP BY age
ORDER BY COUNT(age) DESC
LIMIT 1
Thanks for your help!
Create another table ages to hold the age ranges you are interested in with a field for age_lower, age_upper and a display name age_range such as '2 to 22'
Join the tables with a WHERE clause that puts the age between the lower and upper ranges.
SELECT `age_range`, COUNT(`age`) AS age_count
FROM `member` INNER JOIN `ages`
ON age BETWEEN age_lower AND age_upper
GROUP BY age_range
ORDER BY COUNT(`age`) DESC, `age_range` ASC
SQL Fiddle
This might solve the problem. The only thing I added was a table to hold values 1..x where x is your bucket count. The #T can easily be replaced with your MySQL table name. The results are all possible sets the age falls in, for each age. Then count of how many equal sets.
--IGNORE BUILDING TEST DATA IN SQL SERVER
DECLARE #T TABLE(member INT,age INT)
DECLARE #X INT
SET #X=1
WHILE(#X<=100) BEGIN
INSERT INTO #T SELECT #X, CAST(RAND() * 100 AS INT)
SET #X=#X+1
END
DECLARE #MinAge INT=1
DECLARE #MaxAge INT=100
--YOUR SET TABLE. TO MAKE LIFE EASY YOU NEED A TABLE OF 1..X
DECLARE #SET TABLE (Value INT)
DECLARE #SET_COUNT INT =10
DECLARE #LOOP INT=1
WHILE(#LOOP<=#SET_COUNT) BEGIN
INSERT #SET SELECT #LOOP
SET #LOOP=#LOOP+1
END
SELECT
MinAge,
MaxAge,
SetCount=COUNT(CountFlag)
FROM
(
SELECT
MinAge=AgeMinusSetCount,
MaxAge=AgePlusSetCount,
CountFlag=1
FROM
(
SELECT DISTINCT
ThisAge,
AgeMinusSetCount=(AgeMinusSetCount-1) + Value,
AgePlusSetCount=CASE WHEN (AgeMinusSetCount-1) + Value + #SET_COUNT > #MaxAge THEN #MaxAge ELSE (AgeMinusSetCount-1) + Value + #SET_COUNT END
FROM
(
SELECT
ThisAge=age,
AgeMinusSetCount=CASE WHEN (age - #SET_COUNT) < #MinAge THEN #MinAge ELSE (age) - #SET_COUNT END
FROM
#T
)RANGES
LEFT OUTER JOIN (SELECT Value FROM #SET) AS FanLeft ON 1=1
)AS DETAIL
)AS Summary
GROUP BY
MinAge,
MaxAge
ORDER BY
COUNT(CountFlag) DESC
I have the following function, mysql query:
BEGIN
DECLARE r float(10,2);
DECLARE var_total float(10,2);
DECLARE var_discount float(10,2) DEFAULT null;
SELECT
sum(x.amount)
FROM
(
(SELECT
student_booking_school_course_price as amount
FROM
tbl_student_booking_school_course
WHERE
student_booking_id=par_student_booking_id
)
UNION
(SELECT
student_booking_school_accommodation_price as amount
FROM
tbl_student_booking_school_accommodation
WHERE
student_booking_id=par_student_booking_id
)
UNION
(SELECT
student_booking_school_insurance_price as amount
FROM
tbl_student_booking_school_insurance
WHERE
student_booking_id=par_student_booking_id
)
UNION
(SELECT
student_booking_school_transfer_price as amount
FROM
tbl_student_booking_school_transfer
WHERE
student_booking_id=par_student_booking_id
)
) x
INTO var_total;
IF var_total IS NULL THEN
SET r = 0;
END IF;
-- discount
SET var_discount = (SELECT
sb.student_booking_discount_amount
FROM
tbl_student_booking sb
WHERE
sb.student_booking_id=par_student_booking_id LIMIT 1);
IF var_discount IS NOT NULL THEN
SET r = var_total - var_discount;
end if;
return r;
END
The values are:
9698.88 course
559.55 accommodation
559.55 insurance
145.98 discount
It seems that the first query inside the function, only sums distinct values, as the result with discount is: 10112.45, so is not summing one value of 559.55, I tried to output different things as concat with a string and only see the result as 9698.88course,559.55accommodation, etc.. and it is fine. So I assume the issue is that is not summing if values are equals. The strange thing is that running this from the console, only the query outside the function, it sums ok.
My question is this a normal behaviour of MySql?If so is there a way to prevent this? is this a bug?
What you need here is UNION ALL clause:
SELECT
sum(x.amount)
FROM
(
(SELECT
student_booking_school_course_price as amount
FROM
tbl_student_booking_school_course
WHERE
student_booking_id=par_student_booking_id
)
UNION ALL
(SELECT
student_booking_school_accommodation_price as amount
FROM
tbl_student_booking_school_accommodation
WHERE
student_booking_id=par_student_booking_id
)
UNION ALL
(SELECT
student_booking_school_insurance_price as amount
FROM
tbl_student_booking_school_insurance
WHERE
student_booking_id=par_student_booking_id
)
UNION ALL
(SELECT
student_booking_school_transfer_price as amount
FROM
tbl_student_booking_school_transfer
WHERE
student_booking_id=par_student_booking_id
)
) x
INTO var_total;
The MySQL UNION Documentation says:
A DISTINCT union can be produced explicitly by using UNION DISTINCT or
implicitly by using UNION with no following DISTINCT or ALL keyword.
I want to remove all the non numeric characters from the column. I have bulk data in my database.
Currently I am using method as describe in below link:
http://venerableagents.wordpress.com/2011/01/29/mysql-numeric-functions/
The problem is that its taking too much time for preocessing.
For 1 million of row current logic takes 1 hour to process the data.
please help me..
Thank You,
Ronak
I assume you're doing something like:
update myTable set foo = NumericOnly(foo);
I don't know how much better you can do than that.
One thing that might help a bit, though. In that NumericOnly function, they're doing extra work. I'd remove the SET idx = LENGTH(val)+1; line, since all that will do is start checking the end of the string (the parts we've already checked) again. A string with 5 leading non-numerics would be checked, in full, 5 times.
Removing the line would leave:
DROP FUNCTION IF EXISTS NumericOnly;
CREATE FUNCTION NumericOnly (val VARCHAR(255))
RETURNS VARCHAR(255)
BEGIN
DECLARE idx INT DEFAULT 0;
IF ISNULL(val) THEN RETURN NULL; END IF;
IF LENGTH(val) = 0 THEN RETURN ""; END IF;
SET idx = LENGTH(val);
WHILE idx > 0 DO
IF IsNumeric(SUBSTRING(val,idx,1)) = 0 THEN
SET val = REPLACE(val,SUBSTRING(val,idx,1),"");
END IF;
SET idx = idx - 1;
END WHILE;
RETURN val;
END;
Here's another spin on things...
DEMO: http://sqlfiddle.com/#!2/0c96e/21
First, create yourself a numbers table
CREATE TABLE numbers (
number int NOT NULL PRIMARY KEY
);
INSERT INTO numbers (number)
SELECT n0 + n1 + n2 + n3 + n4 + n5
FROM (SELECT 0 AS n0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3) AS z0
CROSS
JOIN (SELECT 0 AS n1 UNION SELECT 4 UNION SELECT 8 UNION SELECT 12) AS z1
CROSS
JOIN (SELECT 0 AS n2 UNION SELECT 16 UNION SELECT 32 UNION SELECT 48) AS z2
CROSS
JOIN (SELECT 0 AS n3 UNION SELECT 64 UNION SELECT 128 UNION SELECT 192) AS z3
CROSS
JOIN (SELECT 0 AS n4 UNION SELECT 256 UNION SELECT 512 UNION SELECT 768) AS z4
CROSS
JOIN (SELECT 0 AS n5 UNION SELECT 1024 UNION SELECT 2048 UNION SELECT 3072) AS z5
ORDER
BY 1;
Here's some sample data to play with
CREATE TABLE your_table (
foo varchar(50)
);
INSERT INTO your_table (foo)
VALUES ('124nhasfonasf13')
, ('NONE')
, ('r937')
, ('o9o9')
, ('n444n4n455n')
, ('blah');
Then here's a query to give you just the numbers. Should be more efficient as it is SET based instead of iterative like your function example...
SELECT foo
, Group_Concat(c ORDER BY position SEPARATOR '')
FROM (
SELECT vals.foo
, numbers.number As position
, SubString(vals.foo, numbers.number, 1) As c
FROM (
SELECT foo
, Length(foo) As lngth
FROM your_table
WHERE foo REGEXP '[0-9]'
) As vals
INNER
JOIN numbers
ON numbers.number BETWEEN 1 AND vals.lngth
) As x
WHERE c REGEXP '[0-9]'
GROUP
BY foo
Let suppose my table can have values from 000 to 999 (three digits and less than 1000)
Some of this values are filled. Let's suppose currently my table has
000,002,005,190 (001,004,003,006,..189,191,..,999 can be inserted into table)
and these values are randomly allocated 000 and 002 is in table but 001 is not in table yet.
How can I get the values that I can insert into table yet.
DECLARE #t TABLE
(VALUE CHAR(3))
INSERT #t
VALUES
('000'),('002'),('005'),('190')
;WITH rnCTE
AS
(
SELECT -1 + ROW_NUMBER() OVER (ORDER BY TYPE, number, name) AS rn
FROM master.dbo.spt_values
)
SELECT RIGHT('000' + CAST( rn AS VARCHAR(11)),3)
FROM rnCTE
WHERE NOT EXISTS ( SELECT 1 FROM #t
WHERE VALUE = rn
)
AND rn < 1000
EDIT
This query works by generating the complete list of possible numbers from a system table (master.dbo.spt_values) which is guaranteed to contain more than 1000 rows inside the CTE rnCTE. -1 is added to ROW_NUMBER to have the values start at 0 rather than 1.
The outer query zero pads the numbers for display, returning only those which are not in the source data and are less than 1000.
DECLARE #t TABLE(id INT)
INSERT INTO #t (id)
VALUES
(1),(19),(3)
;WITH numbers AS (
SELECT ROW_NUMBER() OVER(ORDER BY o.object_id,o2.object_id) RN FROM sys.objects o
CROSS JOIN sys.objects o2
), NotExisted AS(
SELECT * FROM numbers WHERE RN NOT IN (SELECT ID FROM #t)
AND RN<1000)
SELECT TOP 1 RN FROM NotExisted ORDER BY NEWID()
You will have to write a T-SQL to first query and find the gaps. There is no ready made SQL that will give you the gaps directly.