I am receiving JSON data to my server from each client. I have three main tables; datatypes, templaricustomers and mqttpacket.
Here the datatypes are coming from JSON variable names and I am keeping them in the database.
As I am a beginner in MySQL, I am trying to make a loop and insert the parsed JSON to related tables.
CREATE DEFINER=`root`#`localhost` PROCEDURE `SP_INSERT_DATA`(
IN `incoming_data` TEXT,
IN `value_array` TEXT,
IN `customer_id` INT
)
LANGUAGE SQL
NOT DETERMINISTIC
CONTAINS SQL
SQL SECURITY DEFINER
COMMENT ''
BEGIN
DECLARE i INT;
DECLARE value_iteration VARCHAR(50);
DECLARE lcl_data_type_id INT;
SET i = 1;
WHILE (LOCATE(',', value_array) > 0)
DO
SET #arr_data_type_name = SUBSTRING_INDEX(value_array,',',i);
SET value_array = SUBSTRING(value_array, LOCATE(',',value_array) + 1);
SELECT JSON_EXTRACT(#incoming_data, #arr_data_type_name) INTO value_iteration;
SET #arr_data_type_name := SUBSTRING_INDEX(#arr_data_type_name, ".", -1);
SELECT id INTO lcl_data_type_id FROM test_database.datatypes WHERE datatypes.data_name = #arr_data_type_name LIMIT 1;
INSERT INTO test_database.mqttpacket (data_type_id,inserted_time,customer_id,data_value) VALUES(lcl_data_type_id,NOW(),customer_id,value_iteration);
SET i = i+1;
END WHILE;
END
Example incoming_data in JSON is like;
{"d": {"subcooling": 6,"B1": 382,"B2": 386,"B3": 526,"B4": 361,"B5": 713,"B6": 689,"B7": 386,"B8": 99,"Discharge": 663,"Suction": 111,"High_Pressure": 225,"Low_Pressure": 78,"Evaporation": 31,"Condensation": 388,"MAX_CMP_SPEED": 950,"Thermal_Limit": 950,"SH": 78,"EEV_pct": 571,"COP": 52,"DSH": 272,"Water Flux": 713,"Fan Power": 239,"Delta T to Start": 0,"Delta P to Start": 60,"CMP_ROTOR_RPS": 430,"SET_CH_FLASH": 120,"SET_HP_FLASH": 500,"SET_DHW_FLASH": 500,"Defrosting": 0,"B8_AVERAGE": 42,"SET_PLANT": 0,"SET_CH_BMS": 430,"SET_HP_BMS": 382,"SET_DHW_BMS": 510,"SET_ACTIVE": 402,"SET_DSH": 323,"EEV_INJ_pct": 0,"LPT": 0,"HPT": 0,"PLANT_MODE_MANUAL": 0,"DHW_MODE_MANUAL": 0,"WATER_FLOW": 713,"DISCHARGE_TMP": 663,"INVERTER_TMP": 25,"ENVELOP_ZONE": 1,"EEV_A_STEPS": 274,"EBM_POWER": 239,"EBM_MAX_POWER": 322,"COMP_pct_FINAL": 359,"TOTAL_POWER_ABSORBED": 2599,"NAME": [17236,11585,13388,50,0,0,0,0,0,0,0,0,0,0,0,0],"POWER_OUT_KW": 134,"COOLING CAPACITY": [35],"EBM1_PCT": [861],"EBM2_PCT": [767]},"ts": "2021-02-02T14:42:02.479731" }
An example of value_array is like;
$.d.subcooling,$.d.B1,$.d.B2
This is my Stored Procedure. I just need to extract the JSON node by node and find the "datatypename" which is "node name" from "incoming_data" and insert into mqtt_packet table by it's value..
It's not able to fetch the data which is "value_iteration" and inserts unrelated data type ids..
Please advise me what is wrong with my query.
I hope I was clear... Cheers!
Related
I need to receive a json string as a parameter of the procedure and parse it as a json object.
An example of this json string: {'new_settings': [{'setting_name': 'test', 'setting_value': 'test_value'}]}
I need to fetch the array property named "new_settings" and iterate through the objects inside this array.
This is what i have at the moment:
CREATE PROCEDURE `sp_test` (IN `_settings` longtext)
BEGIN
DECLARE i INT UNSIGNED DEFAULT 0;
DECLARE count INT UNSIGNED DEFAULT 0;
DECLARE current_item LONGTEXT;
START TRANSACTION;
SET count = (JSON_LENGTH(JSON_EXTRACT(_settings, '$.new_settings')) - 1);
WHILE i < count DO
SET current_item := (_settings, CONCAT('$[', i, ']'));
INSERT INTO settings(setting_name, setting_value) VALUES (JSON_EXTRACT(current_item, '$.setting_name'), JSON_EXTRACT(current_item, '$.setting_value'));
SET i := i + 1;
END WHILE;
COMMIT;
END
But i am getting the following error: "Syntax error in JSON text in argument 1 to function 'json_extract' at position 3" and i dont understand what is wrong.
Can you provide some insight?
Thank you in advance.
Your json is all wrong.
You always should check with json Validators, if mysql can read your input.
SET #a = '{"new_settings": [{"setting_name": "test", "setting_value": "test_value"}]}';
SELECT JSON_EXTRACT(#a, '$.new_settings[0].setting_name') c1
, JSON_EXTRACT(#a, '$.new_settings[0].setting_value') c2;
Would give you
c1 c2
test test_value
Where did you get your json String?
I am Using Mysql DB. my question is : can i create array using sql?
if Yes then how and how to populate this array with output of following query -- "Select column_name1 From tableName".
Help me, Thanks in Advance
As I mentioned in my comment, MySQL does not support arrays by itself. That kind of structures are supported by other programming languages (like PHP, Java, Python, etcetera) and you can write a program capable of connecting to a MySQL database, read data from it and populate arrays (I think PostgreSQL supports an array data type, but I'm not sure).
What you can do is use cursors in a stored procedure to retreive data from a query and store it into variables.
Example:
delimiter $$
create procedure my_procedure()
begin
declare value varchar(100);
declare done int default false;
declare cur cursor for
select column_name1 from your_table;
declare continue handler for not found set done = true;
open cur; -- This line will open the row set and place the cursor
-- on the first row.
loop_data: loop
fetch cur into value; -- This line will fetch the current row
-- into the variable and move the cursor
-- to the next row.
if done then -- If there are no more rows in the
leave loop_data; -- row set, the loop is terminated here
end if; -- and the execution moves to the next
-- instruction after "end loop;"
-- do whatever you need to do with the retrieved value
end loop;
close cur;
end $$
delimiter ;
If you want to use an array in a high level programming language, you can do it using the appropriate methods. Here's an example using Java (read The Java tutorials: JDBC Database access for more info):
public class SomeClass {
/*
Retrieve data from a database and return an array with it.
Parameters:
- conn: Connection to the database.
*/
public String[] getValues(Connection conn) {
String[] ans = new String[10];
int i;
try(
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery(
"select column_name1 from your_table limit 10"
);
) {
rs.beforeFirst();
i = 0;
while(rs.next()) {
ans[i] = rs.getString("column_name1");
i++;
}
} catch(SQLException e) {
// Code to handle the SQL exception
}
return ans;
}
}
References:
MySQL reference manual: Cursors
You can use an variable to simply select your row values into a string. Not precisely an array, but it allows you to store all your values into a single variable:
-- load test data
create table tableName (column_name1 varchar(5));
insert into tableName values
('abcde');
insert into tableName values
('fghij');
insert into tableName values
('klmno');
insert into tableName values
('pqrst');
insert into tableName values
('uvwzy');
insert into tableName values
('z');
-- build "array"
set #array := '';
select #array := concat(#array,column_name1) as array
from tableName;
select #array;
I am in the process of converting a SQL Server 2005 database to MySQL and having problems with a Stored procedure. I'm new to MySQL stored procedures so I'm sure it is a problem with my conversion but I'm not seeing it.
The stored procedure is supposed to generate a temporary table which is used to populate a Data Grid View in a vb.net application. However, I'm getting the error "Data No Data - Zero rows fetched, selected or processed.". Seems simple enough but the select procedure in the stored procedure will get data if I just run it as a query which is why I don't understand why the error.
I'm really hoping someone can tell me why because I have several hundred stored procedures to convert and I'm having this problem on the very first one.
Here's the Stored Procedure:
DELIMITER $$
DROP PROCEDURE IF EXISTS `usp_get_unassigned_media`$$
CREATE DEFINER=`showxx`#`67.111.11.110` PROCEDURE `usp_get_unassigned_media`()
BEGIN
/* GET CURSOR WITH LOCAL LOCATIONS */
DECLARE intKey INT;
DECLARE dteDateInserted DATETIME;
DECLARE vchIdField VARCHAR(200);
DECLARE vchValueField VARCHAR(200);
DECLARE intLastKey INT;
/*TAKE OUT SPECIFIC PLAYLIST ITEMS IF TOO SLOW*/
DECLARE csrMediaToBeAssigned CURSOR FOR
SELECT
`media`.`key` AS `key`,
`media`.`date_inserted` AS `date_inserted`,
`media_detail_types`.`id` AS `id`,
`media_details`.`value` AS `value`
FROM (`media`
LEFT JOIN (`media_detail_types`
JOIN `media_details`
ON ((`media_detail_types`.`key` = `media_details`.`detail_key`)))
ON ((`media_details`.`media_key` = `media`.`key`)))
WHERE ((`media`.`is_assigned` = 0)
AND ((`media_detail_types`.`id` = 'Volume Name')
OR (`media_detail_types`.`id` = 'Drive Id')))
ORDER BY `media`.`key`,`media`.`date_inserted`,`media_detail_types`.`id`;
OPEN csrMediaToBeAssigned;
DROP TEMPORARY TABLE IF EXISTS temp_unassigned_media;
CREATE TEMPORARY TABLE temp_unnassigned_media
(temp_key INT, DateInserted DATETIME, IdField VARCHAR(200), ValueField VARCHAR (200))
ENGINE=MEMORY;
SET intLastKey = 0;
/*--GET FIRST RECORD */
FETCH FROM csrMediaToBeAssigned
INTO intKey, dteDateInserted, vchIdField, vchValueField;
/*--LOOP THROUGH CURSOR */
WHILE intLastKey = 0 DO
/*--DATA SHOULD BE IN DRIVE ID THEN VOLUME NAME */
INSERT INTO temp_unnassigned_media
VALUES (intKey, dteDateInserted, vchValueField, '');
FETCH NEXT FROM csrMediaToBeAssigned
INTO intKey, dteDateInserted, vchIdField, vchValueField;
UPDATE temp_unnassigned_media
SET IdField = vchValueField
WHERE temp_key = temp_key;
FETCH NEXT FROM csrMediaToBeAssigned
INTO intKey, dteDateInserted, vchIdField, vchValueField;
END WHILE;
SELECT *
FROM temp_unnassigned_media
ORDER BY date_inserted;
CLOSE csrMediaToBeAssigned;
/*DEALLOCATE csrMediaToBeAssigned */
/*DROP TABLE #temp_unnassigned_media */
END$$
DELIMITER ;
You never hit a condition where that WHILE loop will exit; you initialize intLastKey variable, but it never changes, so you fetch through the entire resultset. The exception is thrown when you fetch again, after the last record.
The normative pattern is to declare a CONTINUE HANDLER, which MySQL will execute when the NOT FOUND condition is triggered. The handler is normally used to set a variable, which you can then test, so you know when to exit the loop.
In your case, it looks like just adding this line, after your DECLARE CURSOR statement and before the OPEN statement, would be sufficient:
DECLARE CONTINUE HANDLER FOR NOT FOUND SET intLastKey = 1;
I have a table with the following data in one column:
abc,2,2,34,5,3,2,34,32,2,3,2,2
def,2,2,34,5,3,2,34,32,2,3,2,2
I want to take this data and insert it into another table, using the commas as delimiters, just like how you can specify the FIELDTERMINATOR in BULK INSERT statements.
Is there a way to do this using T-SQL?
I'm not sure if there is any direct way to do in the T-SQL , but if you want to use Bulk Insert you can use sqlcmd to export to CSV file and then Import the file back into server using Bulk Insert.
Create a dbo.Split Functionm, you can refer here split string into multiple record
There are tons of good examples.
if you want to execute as batch process, You can execute sqlcmd and 'Bulk Insert'
sqlcmd -S MyServer -d myDB -E -Q "select dbo.Split(col1) from SomeTable"
-o "MyData.csv" -h-1 -s"," -w 700
-s"," sets the column seperator to
bulk insert destTable
from "MyData.csv"
with
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
Otherwise, You can manipulate directly in the T-SQL, but given you have the same identify of columns definition.
INSERT INTO DestinationTable
SELECT dbo.Split(col1) FROM SomeTable
You need to use a Split function to split your string into a table variable, and then insert those values into your table.
There are tons of those split functions out there, with various pros and cons and various number of parameters and so forth.
Here is one that I quite like - very nicely done, clearly explained.
With that function, you should have no trouble converting your column into individual entries for your other table.
EDIT: allow multiple char separators
This is how I solved it, with two functions to do the splitting into columns (if you want a more complete solution with line splitting as well, see my other post here). It involves:
A scalar function (fSubstrNth) for extracting the n-th field of a line, given an separator
A scalar function (fPatIndexMulti) for finding the n-th index of the separator
(Optional) alternative Right function to accept negative values
Finally, some specific code to use in your solution, since SQL doesn't allow dynamic table-function definitions (in other words, you can't SELECT from a function with dynamic columns)
Now, for the code snippets:
fSubstrNth
-- =============================================
-- Author: Bernardo A. Dal Corno
-- Create date: 18/07/2017
-- Description: substring com 2 PatIndex limitando inicio e fim
-- =============================================
CREATE FUNCTION fSubstrNth
(
#Text varchar(max),
#Sep varchar(3),
#N int --Nth campo
)
RETURNS varchar(max)
AS
BEGIN
DECLARE #Result varchar(max)
IF #N<1 RETURN ''
IF #N=1
SET #Result = substring(#Text, 1, dbo.fPatIndexMulti(#Sep,#Text,1)-1)
ELSE
SET #Result = substring(#Text, dbo.fPatIndexMulti(#Sep,#Text,#N-1)+LEN(#Sep), CASE WHEN dbo.fPatIndexMulti(#Sep,#Text,#N)>0 THEN dbo.fPatIndexMulti(#Sep,#Text,#N)-dbo.fPatIndexMulti(#Sep,#Text,#N-1)-LEN(#Sep) ELSE LEN(#Text)+1 END)
RETURN #Result
END
fPatIndexMulti
-- =============================================
-- Author: Bernardo A. Dal Corno
-- Create date: 17/07/2017
-- Description: recursive patIndex
-- =============================================
CREATE FUNCTION [dbo].[fPatIndexMulti]
(
#Find varchar(max),
#In varchar(max),
#N tinyint
)
RETURNS int
AS
BEGIN
DECLARE #lenFind int, #Result int, #Texto varchar(max), #index int
DECLARE #i tinyint=1
SET #lenFind = LEN(#Find)-1
SET #Result = 0
SET #Texto = #In
WHILE (#i <= #N) BEGIN
SET #index = patindex('%'+#Find+'%',#Texto)
IF #index = 0 RETURN 0
SET #Result = #Result + #index
SET #Texto = dbo.xRight(#Texto, (#index + #lenFind)*-1)
SET #i = #i + 1
END
SET #Result = #Result + #lenFind*(#i-2)
RETURN #Result
END
xRight
-- =============================================
-- Author: Bernardo A. Dal Corno
-- Create date: 06/01/2015
-- Description: Right inverso (para nros < 0)
-- =============================================
CREATE FUNCTION [dbo].[xRight]
(
#Texto varchar(8000),
#Qntd int
)
RETURNS varchar(8000)
AS
BEGIN
DECLARE #Result varchar(8000)
IF (Len(#Texto) = 0) OR (#Qntd = 0)
SET #Result = ''
ELSE IF (#Qntd > 0)
SET #Result = Right(#Texto, #Qntd)
ELSE IF (#Qntd < 0)
SET #Result = Right(#Texto, Len(#Texto) + #Qntd)
RETURN #Result
END
Specific code
SELECT
acolumn = 'any value',
field1 = dbo.fSubstrNth(table.datacolumn,',',1),
field2 = dbo.fSubstrNth(table.datacolumn,',',2),
anothercolumn = 'set your query as you would normally do',
field3 = (CASE dbo.fSubstrNth(table.datacolumn,',',3) WHEN 'C' THEN 1 ELSE 0 END)
FROM table
Note that:
fSubstrNth receives the n-th field to extract from the 'datacolumn'
The query can be as any other. This means it can be stored in a procedure, tabled-function, view, etc. You can extract some or all fields, in any order you wish, and process however you want
If used in a stored procedure, you could create a generic way of creating a query and temp table that loads the string with dynamic columns, but you have to make a call to another procedure to use the data OR create a specific query like above in the same procedure (which would make it non-generic, just more reusable)
I want to be able to pass an "array" of values to my stored procedure, instead of calling "Add value" procedure serially.
Can anyone suggest a way to do it? am I missing something here?
Edit: I will be using PostgreSQL / MySQL, I haven't decided yet.
As Chris pointed, in PostgreSQL it's no problem - any base type (like int, text) has it's own array subtype, and you can also create custom types including composite ones. For example:
CREATE TYPE test as (
n int4,
m int4
);
Now you can easily create array of test:
select ARRAY[
row(1,2)::test,
row(3,4)::test,
row(5,6)::test
];
You can write a function that will multiply n*m for each item in array, and return sum of products:
CREATE OR REPLACE FUNCTION test_test(IN work_array test[]) RETURNS INT4 as $$
DECLARE
i INT4;
result INT4 := 0;
BEGIN
FOR i IN SELECT generate_subscripts( work_array, 1 ) LOOP
result := result + work_array[i].n * work_array[i].m;
END LOOP;
RETURN result;
END;
$$ language plpgsql;
and run it:
# SELECT test_test(
ARRAY[
row(1, 2)::test,
row(3,4)::test,
row(5,6)::test
]
);
test_test
-----------
44
(1 row)
If you plan to use MySQL 5.1, it is not possible to pass in an array.
See the MySQL 5.1 faq
If you plan to use PostgreSQL, it is possible look here
I don't know about passing an actual array into those engines (I work with sqlserver) but here's an idea for passing a delimited string and parsing it in your sproc with this function.
CREATE FUNCTION [dbo].[Split]
(
#ItemList NVARCHAR(4000),
#delimiter CHAR(1)
)
RETURNS #IDTable TABLE (Item VARCHAR(50))
AS
BEGIN
DECLARE #tempItemList NVARCHAR(4000)
SET #tempItemList = #ItemList
DECLARE #i INT
DECLARE #Item NVARCHAR(4000)
SET #tempItemList = REPLACE (#tempItemList, ' ', '')
SET #i = CHARINDEX(#delimiter, #tempItemList)
WHILE (LEN(#tempItemList) > 0)
BEGIN
IF #i = 0
SET #Item = #tempItemList
ELSE
SET #Item = LEFT(#tempItemList, #i - 1)
INSERT INTO #IDTable(Item) VALUES(#Item)
IF #i = 0
SET #tempItemList = ''
ELSE
SET #tempItemList = RIGHT(#tempItemList, LEN(#tempItemList) - #i)
SET #i = CHARINDEX(#delimiter, #tempItemList)
END
RETURN
END
You didn't indicate, but if you are referring to SQL server, here's one way.
And the MS support ref.
For PostgreSQL, you could do something like this:
CREATE OR REPLACE FUNCTION fnExplode(in_array anyarray) RETURNS SETOF ANYELEMENT AS
$$
SELECT ($1)[s] FROM generate_series(1,array_upper($1, 1)) AS s;
$$
LANGUAGE SQL IMMUTABLE;
Then, you could pass a delimited string to your stored procedure.
Say, param1 was an input param containing '1|2|3|4|5'
The statement:
SELECT CAST(fnExplode(string_to_array(param1, '|')) AS INTEGER);
results in a result set that can be joined or inserted.
Likewise, for MySQL, you could do something like this:
DELIMITER $$
CREATE PROCEDURE `spTest_Array`
(
v_id_arr TEXT
)
BEGIN
DECLARE v_cur_position INT;
DECLARE v_remainder TEXT;
DECLARE v_cur_string VARCHAR(255);
CREATE TEMPORARY TABLE tmp_test
(
id INT
) ENGINE=MEMORY;
SET v_remainder = v_id_arr;
SET v_cur_position = 1;
WHILE CHAR_LENGTH(v_remainder) > 0 AND v_cur_position > 0 DO
SET v_cur_position = INSTR(v_remainder, '|');
IF v_cur_position = 0 THEN
SET v_cur_string = v_remainder;
ELSE
SET v_cur_string = LEFT(v_remainder, v_cur_position - 1);
END IF;
IF TRIM(v_cur_string) != '' THEN
INSERT INTO tmp_test
(id)
VALUES
(v_cur_string);
END IF;
SET v_remainder = SUBSTRING(v_remainder, v_cur_position + 1);
END WHILE;
SELECT
id
FROM
tmp_test;
DROP TEMPORARY TABLE tmp_test;
END
$$
Then simply CALL spTest_Array('1|2|3|4|5') should produce the same result set as the above PostgreSQL query.
Thanks to JSON support in MySQL you now actually have the ability to pass an array to your MySQL stored procedure. Create a JSON_ARRAY and simply pass it as a JSON argument to your stored procedure.
Then in procedure, using MySQL's WHILE loop and MySQL's JSON "pathing" , access each of the elements in the JSON_ARRAY and do as you wish.
An example here https://gist.githubusercontent.com/jonathanvx/513066eea8cb5919b648b2453db47890/raw/22f33fdf64a2f292688edbc67392ba2ccf8da47c/json.sql
Incidently, here is how you would add the array to a function (stored-proc) call:
CallableStatement proc = null;
List<Integer> faultcd_array = Arrays.asList(1003, 1234, 5678);
//conn - your connection manager
conn = DriverManager.getConnection(connection string here);
proc = conn.prepareCall("{ ? = call procedureName(?) }");
proc.registerOutParameter(1, Types.OTHER);
//This sets-up the array
Integer[] dataFaults = faultcd_array.toArray(new Integer[faultcd_array.size()]);
java.sql.Array sqlFaultsArray = conn.createArrayOf("int4", dataFaults);
proc.setArray(2, sqlFaultsArray);
//:
//add code to retrieve cursor, use the data.
//: