I'm migrating mysql database to postgres and ran into a roadblock regarding some basic xml functionality. In MySql I had stored procedures which would replace nodes inside xml document but cannot find any way to do so in postgres.
Here's my stored proc from mysql:
CREATE DEFINER=`root`#`localhost` PROCEDURE `SP_UpdateExamFilesXmlNode`(examFileId int, xPathExpression varchar(128), xmlNode longtext)
BEGIN
DECLARE xmlData longtext;
DECLARE newXmlData longtext;
DECLARE xmlNodeCount int;
SET xmlData = NULL;
SELECT xml_data INTO xmlData FROM sonixhub.exam_files WHERE id = examFileId;
IF xmlData IS NOT NULL THEN
-- check if the node already exists and if it does then simply update it
SET xmlNodeCount = ExtractValue(xmlData, CONCAT('count(',xPathExpression,')'));
IF xmlNodeCount > 0 THEN
SET newXmlData = UpdateXML(xmlData, xPathExpression, xmlNode);
-- if node doesn't exist then we have to add it manually
ELSE
SET newXmlData = REPLACE(xmlData, '</ImageXmlData>', CONCAT(xmlNode, '</ImageXmlData>'));
END IF;
UPDATE sonixhub.exam_files SET xml_data = newXmlData WHERE id = examFileId;
ELSE
-- there is no xml found so create xml from scratch and insert the node
SET xmlData = CONCAT('<ImageXmlData>',xmlNode,'</ImageXmlData>');
UPDATE sonixhub.exam_files SET xml_data = xmlData WHERE id = examFileId;
END IF;
END
Is there any way to replicate this functionality in postgres function instead of moving the logic into the application itself?
EDIT - FOUND A SOLUTION TO MY PROBLEM
I found a solution using mix of postgres xml and string formatting functions.
examFileId is used to find the row to be updated with the xml, change the code with your table info
is the hardcoded root node in my case, but you can change it to whatever you like.
Here's how you call the function:
-- this adds <DicomTags> node to your xml value in the table, if <DicomTags> already exists then it's replaced by the one passed in
select update_exam_files_xml_node(1, '/ImageXmlData/DicomTags', '<DicomTags><DicomTag>xxx</DicomTag></DicomTags>');
-- this adds <Settings> node to your xml value in the table, if <Settings> already exists then it's replaced by the one passed in
select update_exam_files_xml_node(1, '/ImageXmlData/Settings', '<Settings>asdf</Settings>');
CREATE OR REPLACE FUNCTION update_exam_files_xml_node(examFileId int, xPathExpression text, xmlNode text)
RETURNS void AS
$BODY$
DECLARE xmlData xml;
DECLARE newXmlData xml;
DECLARE xmlNodeCount int;
DECLARE replaceTag text;
BEGIN
SELECT xml_data INTO xmlData FROM exam_files WHERE id = examFileId;
IF xml_is_well_formed(xmlNode) = false THEN
PERFORM add_error_log('update_exam_files_xml_node', 'xmlNode is not well formed xml');
RETURN;
END IF;
IF xmlData IS NOT NULL THEN
-- check if the node already exists and if it does then simply update it
IF xmlexists(xPathExpression PASSING BY REF xml(xmlData)) = true THEN
-- get the node name
replaceTag := regexp_replace(xPathExpression, '/.*/', '');
-- replace the existing node with the newly passed in node
newXmlData := xml(regexp_replace(xmlData::text, '<'||replaceTag||'>.*</'||replaceTag||'>', xmlNode));
-- if node doesn't exist then we have to add it manually
ELSE
newXmlData := xml(REPLACE(xmlData::text, '</ImageXmlData>', xmlNode||'</ImageXmlData>'));
END IF;
UPDATE exam_files SET xml_data = newXmlData WHERE id = examFileId;
ELSE
-- there is no xml found so create xml from scratch and insert the node
xmlData := '<ImageXmlData>'||xmlNode||'</ImageXmlData>';
UPDATE exam_files SET xml_data = xmlData WHERE id = examFileId;
END IF;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
Glad you got a solution. To be honest, string formatting functions tend to be a bit difficult to reliably use inside SGML due to issues relating to hierarchies of languages. I.e. regexps have hard limits as to what they can do.
A better solution is likely to be to go a very different direction and write your functions in PL/PerlU or PL/Python, and use existing XML processing capabilities for those languages. This is likely to give you a better and more robust solution.
Related
I have a couple of columns that are json arrays that have datetime data like this:
["2017-04-18 11:05:00.000000"]
["2017-04-20 11:05:00.000000"]
["2017-04-22 11:05:00.000000"]
["2017-12-11 22:14:02.000000", "2017-12-11 22:14:08.000000", "2017-12-11 22:19:13.000000", "2017-12-11 22:20:44.000000", "2017-12-11 22:21:54.000000", "2017-12-11 22:23:09.000000"]
["2017-12-13 13:21:04.000000"]
["2017-12-14 13:10:44.000000", "2017-12-14 13:21:51.000000"]
["2017-12-15 13:27:21.000000", "2017-12-15 13:30:21.000000"]
["2017-12-16 15:15:22.000000"]
The goal is to parse out the datetime data and store it into a separate table from which I plan on doing some fun stuff. Currently, it only inserts the first record only, and it inserts it ~180000 times. My current code is:
BEGIN
DECLARE finished INTEGER DEFAULT 0;
DECLARE i INTEGER DEFAULT 0;
DECLARE usages VARCHAR(4000);
-- declare cursor for employee email
DEClARE curUsages
CURSOR FOR
SELECT associated_usages from usagesTbl where associated_usages not like '[]';
-- declare NOT FOUND handler
DECLARE CONTINUE HANDLER
FOR NOT FOUND SET finished = 1;
OPEN curUsages;
getUsages: LOOP
FETCH curUsages INTO usages;
IF finished = 1 THEN
LEAVE getUsages;
END IF;
WHILE i < JSON_LENGTH(usages) DO
INSERT INTO usagesTbl VALUES (JSON_EXTRACT(usages, CONCAT('$[',i,']')));
SET i = i + 1;
END WHILE;
SET i = 0;
END LOOP getUsages;
CLOSE curUsages;
END;
it seems that the while loop variable "i" is not increasing, and I am getting constantly stuck in the loop. The reason for me thinking this is that I pulled out the JSON_EXTRACT code and wrote this for testing:
set #i = 0;
select JSON_EXTRACT(associated_usages, CONCAT('$[',#i,']')) from usagesTbl where associated_usages not like '[]';
I can change the value of #i to whatever index I want and I get the right data. Im just stuck on why it doesn't work in the while loop during the stored procedure. Any help is greatly appreciated!
not sure if this could be the issue, but I see this:
DEClARE curUsages
Should be this:
DECLARE curUsages
Can it be the simple typo ? (the 1 for the L)
Fixed it! It somehow created an infinite loop that just kept on inserting data even when the stored proc said it was done running. I dropped and recreated the table, and changed the datatype of usages back from VARCHAR to json, and it worked like a charm.
Use case:
Application requires a subset of attributes, based on business rules
Example:
Some students do not require to enter in home address
Database : Oracle
Proposed implementation:
Build json object containing all possible attribute named pairs, then selectively remove specific named pairs
Issue:
Hoped to use native oracle function to remove the specified named pair.
e.g json_object.remove_attribute('home_address');
However Oracle do not appear to provide any such method.
Workaround : Convert json_object to VARCHAR2 string, and then use combination of INSTR and REPLACE to remove named pair.
Illustrative code:
DECLARE
CURSOR cur_student_json (p_s_ref IN VARCHAR2) IS
SELECT JSON_OBJECT(
,'s_surname' value s.s_surname
,'s_forename_1' value s.s_forename_1
,'s_home_address_1' value s.s_home_address_1
RETURNING VARCHAR2 ) student_json
FROM students s
WHERE s.s_ref = p_s_ref;
BEGIN
FOR x IN cur_student_json (p_s_ref) LOOP
vs_student_json:=x.student_json;
EXIT;
END LOOP;
-- Determine student type
vs_student_type:=get_student_type(p_s_ref);
-- Collect list of elements not required, based on student type
FOR x IN cur_json_inorout(vs_student_type) LOOP
-- Remove element from the json
vs_student_json:=json_remove(vs_student_json,x.attribute);
END LOOP;
END;
/
Question:
There must be an elegant method to achieve requirement
Classify under RTFM. Needs Oracle 12.2
DECLARE
-- Declare an object of type JSON_OBJECT_T
l_obj JSON_OBJECT_T;
-- Declare cursor to build json object
CURSOR cur_student_json (p_s_ref IN VARCHAR2) IS
SELECT JSON_OBJECT(
,'s_surname' value s.s_surname
,'s_forename_1' value s.s_forename_1
,'s_home_address_1' value s.s_home_address_1
) student_json
FROM students s
WHERE s.s_ref = p_s_ref;
BEGIN
-- Initialise object
l_obj := JSON_OBJECT_T();
-- Populate the object
FOR x IN cur_student_json (p_s_ref) LOOP
l_obj:=JSON_OBJECT_T.parse(x.student_json);
EXIT;
END LOOP;
-- Determine student type
vs_student_type:=get_student_type(p_s_ref);
-- Collect list of elements not required, based on student type
FOR x IN cur_json_inorout(vs_student_type) LOOP
-- Remove element from the json
l_obj.remove(x.attribute);
END LOOP;
-- Display modified object
dbms_output.put_line(l_obj.stringify);
END;
/
I try to do as explained in:
https://wiki.postgresql.org/wiki/Audit_trigger
Auditing values as JSON
For PostgreSQL 9.2, or 9.1 with the fantastic json_91 addon, you can
log the old and new values in the table as structured json instead of
flat text, giving you much more power to query your audit history.
Just change the types of v_old_data, v_new_data, original_data and
new_data from TEXT to json, then replace ROW(OLD.) and ROW(NEW.)
with row_to_json(OLD) and row_to_json(NEW) respectively.
However this get me a error:
CREATE OR REPLACE FUNCTION add_log (name text, Action TEXT, data jsonb, OUT RETURNS BOOLEAN)
AS $$
BEGIN
RETURNS = true;
END;
$$
LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION log_city() RETURNS TRIGGER AS
$$
DECLARE
v_new_data jsonb;
BEGIN
IF (TG_OP = 'UPDATE') THEN
RETURN NEW;
ELSIF (TG_OP = 'INSERT') THEN
v_new_data := row_to_jsonb(NEW);
EXECUTE add_log('City', 'City.New', v_new_data);
RETURN NEW;
END IF;
RETURN NULL; -- result is ignored since this is an AFTER trigger
END;
$$ LANGUAGE plpgsql;
INSERT INTO Location (city, state, country) values ('a', 'b' , 'c')
It say:
ERROR: function row_to_jsonb(location) does not exist
If I put v_new_data := row_to_jsonb(ROW(NEW)); then I get:
ERROR: function row_to_jsonb(record) does not exist
It's stated in the documentation that
Table 9-42 shows the functions that are available for creating json
and jsonb values. (There are no equivalent functions for jsonb, of the
row_to_json and array_to_json functions. However, the to_jsonb
function supplies much the same functionality as these functions
would.)
thus it's row_to_json that has to be used. a row_to_jsonb does not exists but row_to_json produces the desired result for the JSONB type as well.
I have a table name css_diectory with 3 columns directory_id, directory-name and root_directory. I am storing hierarchical directory structure in this table. I have written a procedure to retrieve all the descendants given the directory_id. But it doesn't work. Can anyone please help me.
Here is the procedure
CREATE PROCEDURE getDescendants
(IN rootId varchar(32), INOUT desecendantsFolderId varchar(3200))
BEGIN
DECLARE endOfRecord INTEGER DEFAULT 0;
DECLARE folderId varchar(32) DEFAULT "";
DECLARE folderName varchar(32) DEFAULT "";
DECLARE folderCursor CURSOR FOR
Select directory_id, directory_name from css_directory where root_directory=rootId;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET endOfRecord = 1;
OPEN folderCursor;
getFolderId: LOOP
FETCH folderCursor INTO folderId, folderName;
IF endOfRecord = 1 THEN
LEAVE getFolderId;
END IF;
Call getDescendants(folderId,desecendantsFolderId);
SET desecendantsFolderId = CONCAT(folderId,",",desecendantsFolderId);
call getDescendants(folderId,desecendantsFolderId);
END LOOP getFolderId;
END
Edit: The output of this procedure is always a null set. It does not produce any error
I'm not 100% sure I follow what you're doing, but it looks like there's a problem here:
SET desecendantsFolderId = CONCAT(folderId,",",desecendantsFolderId);
When any argument to CONCAT() is null, then the return value is null. Presumably descendantsFolderId is initially null, and if so, I don't see where that would change.
There are several ways to remedy this, but here is one of them:
SET desecendantsFolderId = NULLIF(CONCAT_WS(",",folderId,desecendantsFolderId),"");
CONCAT_WS() is like CONCAT(), except the first argument is used as a separator amd null arguments beyond the first one are disregarded, and empty string is returned if all subsequent arguments are null. NULLIF() is probably not technically needed, based on the rest of the code, but will ensure that the final result of CONCAT_WS() will be turned back to null if the input args are indeed all null.
What would the XQuery look like to check if a node exists, and if it does then run a replace statement, if not then an insert statement?
Here's what I have in mind. I want to store whether or not a user has read an important message in XML. Here's what the data would look like.
<usersettings>
<message haveRead="0" messageId="23" ></message>
<message haveRead="1" messageId="22" ></message>
</usersettings>
Basically this XML tells me that the user has read one message, while the other message still needs to be viewed / read.
I want to combine my insert / replace xquery into one statement. Here's what I had in mind.
UPDATE WebUsers SET UserSettings.modify('
declare default element namespace "http://www.test.com/test";
IF a node exists with the messageId
code to replace node with new update
ELSE
code to insert a new node with the provided variables
')
WHERE Id = #WebUserId
I haven't found a really satisfactory way of doing this, but one of these might work for you. I like the first technique better, but I'm annoyed that I haven't found a more elegant way of doing it.
Make sure the node always exists first
DECLARE #messageID int;
SET #messageID=24;
DECLARE #myDoc xml;
SET #myDoc =
'<usersettings>
<message haveRead="0" messageId="23" >msg</message>
<message haveRead="1" messageId="22" >msg</message>
</usersettings>';
SELECT #myDoc;
SET #myDoc.modify('
insert
if (count(//message[#messageId=sql:variable("#messageID")]) = 0)
then <message haveRead="0">new msg</message>
else()
as last into (/usersettings)[1]
');
SELECT #myDoc;
--now do the rest, safe that the node exists
Switching
DECLARE #myDoc xml;
SET #myDoc =
'<usersettings>
<message haveRead="0" messageId="23" >msg</message>
<message haveRead="1" messageId="22" >msg</message>
</usersettings>';
SELECT #myDoc;
DECLARE #messageID int;
SET #messageID=23;
IF #myDoc.exist('//message[#messageId=sql:variable("#messageID")]') = 1
BEGIN
SET #myDoc.modify('replace value of (//message[#messageId=sql:variable("#messageID")]/text())[1]
with "test"')
END
ELSE
BEGIN
SET #myDoc.modify('insert <message haveRead="0">new msg</message>
into (/usersettings)[1]')
END
SELECT #myDoc;