while converting Json to table in Mysql having 50000 line item... It take long time to execute (20 mins).Below is patch of code... Please help me...
Set I_InvoiceItemDetails = JSON_UNQUOTE(JSON_EXTRACT(p_data, '$.InvoiceItemDetails'));
While (J < JSON_Length(I_InvoiceItemDetails)) DO
SET B = JSON_UNQUOTE(JSON_EXTRACT(I_InvoiceItemDetails, CONCAT('$[', J, ']')));
SET I_InvoiceNo = JSON_UNQUOTE(JSON_EXTRACT(B, '$.InvoiceNo'));
Insert Into temp_InvoiceitemDetailsData (_InvoiceNo)
Values (I_InvoiceNo)
Select J + 1 Into J;
End While;
Related
I met a problem when calling a user-defined function in MySQL. The computation is very simple but can't grasp where it went wrong and why it went wrong. Here's the thing.
So I created this function:
DELIMITER //
CREATE FUNCTION fn_computeLoanAmortization (_empId INT, _typeId INT)
RETURNS DECIMAL(17, 2)
BEGIN
SET #loanDeduction = 0.00;
SELECT TotalAmount, PeriodicDeduction, TotalInstallments, DeductionFlag
INTO #totalAmount, #periodicDeduction, #totalInstallments, #deductionFlag
FROM loans_table
WHERE TypeId = _typeId AND EmpId = _empId;
IF (#deductionFlag = 1) THEN
SET #remaining = #totalAmount - #totalInstallments;
IF(#remaining < #periodicDeduction) THEN
SET #loanDeduction = #remaining;
ELSE
SET #loanDeduction = #periodicDeduction;
END IF;
END IF;
RETURN #loanDeduction;
END;//
DELIMITER ;
If I call it like this, it works fine:
SELECT fn_computeLoanAmortization(3, 4)
But if I call it inside a SELECT statement, the result becomes erroneous:
SELECT Id, fn_computeLoanAmortization(Id, 4) AS Amort FROM emp_table
There's only one entry in the loans_table and the above statement should only result with one row having value in the Amort column but there are lots of random rows with the same Amort value as the one with the matching entry, which should not be the case.
Have anyone met this kind of weird dilemma? Or I might have done something wrong from my end. Kindly enlighten me.
Thank you very much.
EDIT:
By erroneous, I meant it like this:
loans_table has one record
EmpId = 1
TypeId = 2
PeriodicDeduction = 100
TotalAmount = 1000
TotalInstallments = 200
DeductionFlag = 1
emp_table has several rows
EmpId = 1
Name = Paolo
EmpId = 2
Name = Nikko
...
EmpId = 5
Name = Ariel
when I query the following statements, I get the correct value:
SELECT fn_computeLoanAmortization(1, 2)
SELECT Id, fn_computeLoanAmortization(Id, 2) AS Amort FROM emp_table WHERE EmpId = 1
But when I query this statement, I get incorrect values:
SELECT Id, fn_computeLoanAmortization(Id, 2) AS Amort FROM emp_table
Resultset would be:
EmpId | Amort
--------------------
1 | 100
2 | 100 (this should be 0, but the query returns 100)
3 | 100 (same error here)
...
5 | 100 (same error here up to the last record)
Inside your function, the variables you use to retrieve the values from the loans_table table are not local variables local to the function but session variables. When the select inside the function does not find any row, those variables still have the same values as from the previous execution of the function.
Use real local variables instead. In order to do that, use the variables names without # as a prefix and declare the variables at the beginning of the function. See this answer for more details.
I suspect the problem is that the variables in the INTO are not re-set when there is no matching row.
Just set them before the INTO:
BEGIN
SET #loanDeduction = 0.00;
SET #totalAmount = 0;
SET #periodicDeduction = 0;
SET #totalInstallments = 0;
SET #deductionFlag = 0;
SELECT TotalAmount, PeriodicDeduction, TotalInstallments, DeductionFlag
. . .
You might just want to set them to NULL.
Or, switch your logic to use local variables:
SET v_loanDeduction = 0.00;
SET v_totalAmount = 0;
SET v_periodicDeduction = 0;
SET v_totalInstallments = 0;
SET v_deductionFlag = 0;
And so on.
I have added the index to my update query and by adding the same query start taking several hours to complete the process.While without index its completing in some minutes i have added the index to faster the process but it became very slow exactly opposite to my desire.
Below is sample code snippet of my code.
Cursor c_updt_stg_rsn is
select distinct substr(r.state_claim_id, 1, 12), ROWID
from nemis.stg_state_resp_rsn r
WHERE r.seq_resp_plan_id = v_seq_resp_plan_id
and r.submitted_claim_id is null
and r.filler_2 is null;
BEGIN
OPEN c_updt_stg_rsn;
LOOP
FETCH c_updt_stg_rsn BULK COLLECT
INTO v_state_claim_id, v_rowid LIMIT c_BULK_SIZE;
FORALL i IN 1 .. v_state_claim_id.COUNT()
UPDATE /*+ index(STG_STATE_RESP_RSN,IDX2_STG_STATE_RESP_RSN) */ nemis.stg_state_resp_rsn
SET (submitted_claim_id , filler_2) = (SELECT DISTINCT submitted_claim_id, sl_group_id FROM nemis.state_sub_Resp_dtl D WHERE
(d.state, d.type_of_claim) in (select distinct state, type_of_claim
from nemis.resp_match_state
where seq_resp_match_table_level_id in
(select seq_resp_match_table_level_id
from nemis.resp_match_table_level
where seq_resp_plan_id = v_seq_resp_plan_id))
AND resp_state_claim_id LIKE v_state_claim_id(i)||'%'
)
WHERE ROWID = v_rowid(i);
IF v_state_claim_id.COUNT() != 0 THEN
v_cnt_rsn := v_cnt_rsn + SQL%ROWCOUNT;
END IF;
COMMIT;
EXIT WHEN c_updt_stg_rsn%NOTFOUND;
END LOOP;
CLOSE c_updt_stg_rsn;
I have a stored procedure which I'm trying to call, and it takes forever to execute. I have no idea what's wrong. A similar stored procedure in another database executes perfectly. I'm not well-versed with MySQL Workbench, so I don't know if the database settings are different or something.
Following is my stored procedure:
CREATE
DEFINER = `admin`#`%`
PROCEDURE `calculate_daily_coil_moved_by_crane_data`()
BEGIN
set #curr_date = curdate();
set #pre_date = date_add(curdate(), interval -1 day);
set #a_shift_start_ts = concat(#pre_date, ' 06:00:00');
set #a_shift_end_ts = concat(#pre_date, ' 13:59:59');
set #b_shift_start_ts = concat(#pre_date, ' 14:00:00');
set #b_shift_end_ts = concat(#pre_date, ' 21:59:59');
set #c_shift_start_ts = concat(#pre_date, ' 22:00:00');
set #c_shift_end_ts = concat(#curr_date, ' 05:59:59');
SELECT #curr_date,
#pre_date,
#a_shift_start_ts,
#a_shift_end_ts,
#b_shift_start_ts,
#b_shift_end_ts,
#c_shift_start_ts,
#c_shift_end_ts;
#SET DATA
insert into daily_coil_move_by_crane_data_for_report (crane_id, crane_name, date, a_shift, b_shift, c_shift)
select cr.id, cr.name, #pre_date, 0, 0, 0
from yms_phase3.crane cr
where active = 1
order by cr.name;
#----------------------------------------------------------------------------------------------------
#--> COILS MOVED BY CRANE A Shift <--
#----------------------------------------------------------------------------------------------------
SET #shift = 'A';
#FETCH ROW DATA
update daily_coil_move_by_crane_data_for_report
set a_shift = ifnull((select COUNT(*)
FROM yms_phase3.workorder_history in_data
where in_data.crane_id = daily_coil_move_by_crane_data_for_report.crane_id
and current_execution_status IN (6 , 7)
and in_data.pick_ts between #a_shift_start_ts and #a_shift_end_ts
group by in_data.crane_name), 0)
where (a_shift is null or a_shift = 0);
#----------------------------------------------------------------------------------------------------
#--> COILS MOVED BY CRANE B Shift <--
#----------------------------------------------------------------------------------------------------
SET #shift = 'B';
#FETCH ROW DATA
update daily_coil_move_by_crane_data_for_report
set b_shift = ifnull((select COUNT(*)
FROM yms_phase3.workorder_history in_data
where in_data.crane_id = daily_coil_move_by_crane_data_for_report.crane_id
and current_execution_status IN (6 , 7)
and in_data.pick_ts between #b_shift_start_ts and #b_shift_end_ts
group by in_data.crane_name), 0)
where (b_shift is null or b_shift = 0);
#----------------------------------------------------------------------------------------------------
#--> COILS MOVED BY CRANE C Shift <--
#----------------------------------------------------------------------------------------------------
SET #shift = 'C';
#FETCH ROW DATA
update daily_coil_move_by_crane_data_for_report
set c_shift = ifnull((select COUNT(*)
FROM yms_phase3.workorder_history in_data
where in_data.crane_id = daily_coil_move_by_crane_data_for_report.crane_id
and current_execution_status IN (6 , 7)
and in_data.pick_ts between #c_shift_start_ts and #c_shift_end_ts
group by in_data.crane_name), 0)
where (c_shift is null or c_shift = 0);
#----------------------------------------------------------------------------------------------------
#INSERT ALL CRANE ENTRY
insert into daily_coil_move_by_crane_data_for_report (crane_id, crane_name, date, a_shift, b_shift, c_shift)
select -1, 'ALL', #pre_date, SUM(a_shift), sum(b_shift), sum(c_shift)
from daily_coil_move_by_crane_data_for_report
where date = #pre_date
group by date;
#UPDATE TOTAL
update daily_coil_move_by_crane_data_for_report
set total_coils_moved = (a_shift + b_shift + c_shift)
where date = #pre_date;
END
Also tried to execute the query from Java using the following:
jdbcTemplate.execute("CALL calculate_daily_coil_moved_by_crane_data;");
But it gives me the following Exception:
java.sql.SQLException: Lock wait timeout exceeded
Any workaround I can do to solve this?
Please try and edit the configuration file, also search for the same here on stack. There are certain possibilities while checking this out,
Check and edit the config file on Hard drive for MySQL increase the cache capacity and default values as the default values are in KB's the memory allocated is very less and to execute such a big procedure it should at least be some MB.
Increase the connection String timeout, that is by setting up right time in seconds. by default it is 60 seconds, which is very less for executing such a procedure, I think in c# at least we set it to '0' seconds which means that it shall not get timed-out till the query is executed.
If Any left Joins/ inner query please try and check whether the same output is produced in inner joins ? as inner joins are faster than left or right joins.
Add Indexes, have foreign key references properly mapped for faster execution of query.
Hope it works.
I need to convert an adjacency list to nested set in MySql. I have found only one resource over the internet to convert an adjacency list into a nested set using mysql(http://data.bangtech.com/sql/nested_set_treeview.htm). The code is also on the same webpage.
CREATE TABLE test.Tree
(emp CHAR(10) NOT NULL,
boss CHAR(10));
CREATE TABLE test.Personnel(
emp CHAR(20) PRIMARY KEY,
boss CHAR(20) REFERENCES Personnel(emp),
salary DECIMAL(6,2) NOT NULL
);
INSERT INTO test.Personnel VALUES ('jerry', 'NULL',1000.00);
INSERT INTO test.Personnel VALUES ('Bert', 'jerry',900.00);
INSERT INTO test.Personnel VALUES ('chuck', 'jerry',900.00);
INSERT INTO test.Personnel VALUES ('donna', 'chuck',800.00);
INSERT INTO test.Personnel VALUES ('eddie', 'chuck',700.00);
INSERT INTO test.Personnel VALUES ('fred', 'chuck',600.00);
INSERT INTO test.Tree
SELECT emp, boss FROM test.Personnel;
I make the Tree table from Personnel table. Tree table has the boss-employee hierarchy. This is an adjacency list. To convert it to the nest set, I applied this code.
BEGIN ATOMIC
DECLARE counter integer;
DECLARE max_counter integer;
DECLARE current_top integer;
SET counter = 2;
SET max_counter = 2 * (SELECT COUNT(*) FROM test.Tree);
SET current_top = 1;
INSERT INTO test.Stack
SELECT 1, emp, 1, NULL
FROM test.Tree
WHERE boss IS NULL;
DELETE FROM test.Tree
WHERE boss IS NULL;
WHILE counter <=(max_counter - 2)
LOOP IF EXISTS (SELECT * FROM test.Stack AS S1, test.Tree AS T1
WHERE S1.emp = T1.boss AND S1.stack_top = current_top)
THEN
BEGIN -- push when top has subordinates, set lft value
INSERT INTO test.Stack
SELECT (current_top + 1), MIN(T1.emp), counter, NULL
FROM test.Stack AS S1, test.Tree AS T1
WHERE S1.emp = T1.boss
AND S1.stack_top = current_top;
DELETE FROM test.Tree
WHERE emp = (SELECT emp
FROM test.Stack
WHERE stack_top = current_top + 1);
SET counter = counter + 1;
SET current_top = current_top + 1;
END
ELSE
BEGIN -- pop the stack and set rgt value
UPDATE test.Stack
SET rgt = counter,
stack_top = -stack_top -- pops the stack
WHERE stack_top = current_top
SET counter = counter + 1;
SET current_top = current_top - 1;
END IF;
END LOOP;
END;
MySQL workbench shows several syntax errors which I could not remove.
I am familiar with only very basic operations of mysql so could not debug the code on my own. How to remove all these errors? Plz Help.
The second source I found to do the above operation is http://www.sqlservercentral.com/articles/Hierarchy/94040/ but the code is in T Sql and I don't have enough skills to translate it to MySQL.
You should put NULL and not 'NULL' in the line
INSERT INTO test.Personnel VALUES ('jerry', 'NULL',1000.00);
Correct version:
INSERT INTO test.Personnel VALUES ('jerry', NULL, 1000.00);
Use Bash:
Bash converting:
# SQL command to fetch necessary fields, output it to text archive "tree"
SELECT id, parent_id, name FROM projects;
# Make a list "id|parentid|name" and sort by name
cat tree |
cut -d "|" -f 2-4 |
sed 's/^ *//;s/ *| */|/g' |
sort -t "|" -k 3,3 > list
# Creates the parenthood chain on second field
while IFS="|" read i p o
do
l=$p
while [[ "$p" != "NULL" ]]
do
p=$(grep -w "^$p" list | cut -d "|" -f 2)
l="$l,$p"
done
echo "$i|$l|$o"
done < list > listpar
# Creates left and right on 4th and 5th fields for interaction 0
let left=0
while IFS="|" read i l o
do
let dif=$(grep "\b$i,NULL|" listpar | wc -l)*2+1
let right=++left+dif
echo "$i|$l|$o|$left|$right"
let left=right
done <<< "$(grep "|NULL|" listpar)" > i0
# The same for following interactions
n=0
while [ -s i$n ]
do
while IFS="|" read i l nil left nil
do
grep "|$i,$l|" listpar |
while IFS="|" read i l o
do
let dif=$(grep "\b$i,$l|" listpar | wc -l)*2+1
let right=++left+dif
echo "$i|$l|$o|$left|$right"
let left=right
done
done < i$n > i$((++n))
done
# Show concatenated
cat i*|sort -t"|" -k 4n
# SQL commands
while IFS="|" read id nil nil left right
do
echo "UPDATE projects SET lft=$left, rgt=$right WHERE id=$id;"
done <<< "$(cat i*)"
I like to return only the first number of a text stored in a column of a database table.
User have put in page ranges into a field like 'p.2-5' or 'page 2 to 5' or '2 - 5'.
I am interested in the '2' here.
I tried to
SELECT SUBSTR(the_field, LOCATE('2', the_field, 1)) AS 'the_number'
FROM the_table
and it works. But how to get ANY number?
I tried
SELECT SUBSTR(the_field, LOCATE(REGEXP '[0-9], the_field, 1)) AS 'the_number'
FROM the_table
but this time I get an error.
Any ideas?
Just use REGEXP_SUBSTR():
SELECT REGEXP_SUBSTR(`the_field`,'^[0-9]+') AS `the_number` FROM `the_table`;
Notes:
I'm using MySQL Server v8.0.
This pattern assumes that the_field is trimmed. Otherwise, use TRIM() first.
REGEXP is not a function in MySQL, but something of an operator. Returns 1 if field matches the regular expression, or 0 if it does not. You cannot use it to figure out a position in a string.
Usage:
mysql> SELECT 'Monty!' REGEXP '.*';
-> 1
As for answer to the question: I don't think there is a simple way to do that using MySQL only. You would be better off processing that field in the code, or extract values before inserting.
For the specific case in the question. Where the String is {number}{string}{number}
there is a simple solution to get the first number. In our case we had numbers like 1/2,3
4-10
1,2
and we were looking for the first number in each row.
It turned out that for this case one can use convert function to convert it into number. MySQL will return the first number
select convert(the_field ,SIGNED) as the_first_number from the_table
or more hard core will be
SELECT
the_field,
#num := CONVERT(the_field, SIGNED) AS cast_num,
SUBSTRING(the_field, 1, LOCATE(#num, the_field) + LENGTH(#num) - 1) AS num_part,
SUBSTRING(the_field, LOCATE(#num, the_field) + LENGTH(#num)) AS txt_part
FROM the_table;
This was original post at source by Eamon Daly
What does it do?
#num := CONVERT(the_field, SIGNED) AS cast_num # try to convert it into a number
SUBSTRING(the_field, 1, LOCATE(#num, the_field) + LENGTH(#num) - 1) # gets the number by using the length and the location of #num in field
SUBSTRING(the_field, LOCATE(#num, the_field) + LENGTH(#num)) # finds the rest of the string after the number.
Some thoughts for future use
Its worth keeping another column which will hold the first number after you parsed it before insert it to the database. Actually this is what we are doing these days.
Edit
Just saw that you have text like p.2-5 and etc.. which means the above cannot work as if the string does not start with a number convert return zero
There's no built-in way that I know of, but here's a Mysql function you can define, this will do it (I didn't code for minus-signs or non-integers, but those could of course be added).
Once created, you can use it like any other function:
SELECT firstNumber(the_field) from the_table;
Here's the code:
DELIMITER //
CREATE FUNCTION firstNumber(s TEXT)
RETURNS INTEGER
COMMENT 'Returns the first integer found in a string'
DETERMINISTIC
BEGIN
DECLARE token TEXT DEFAULT '';
DECLARE len INTEGER DEFAULT 0;
DECLARE ind INTEGER DEFAULT 0;
DECLARE thisChar CHAR(1) DEFAULT ' ';
SET len = CHAR_LENGTH(s);
SET ind = 1;
WHILE ind <= len DO
SET thisChar = SUBSTRING(s, ind, 1);
IF (ORD(thisChar) >= 48 AND ORD(thisChar) <= 57) THEN
SET token = CONCAT(token, thisChar);
ELSEIF token <> '' THEN
SET ind = len + 1;
END IF;
SET ind = ind + 1;
END WHILE;
IF token = '' THEN
RETURN 0;
END IF;
RETURN token;
END //
DELIMITER ;