We have a monitor on our databases to check for ids approaching max-int or max-bigint. We just moved from MySQL, and I'm struggling to get a similar check working on PostgreSQL. I'm hoping someone can help.
Here's the query in MySQL
SELECT table_name, auto_increment FROM information_schema.tables WHERE table_schema = DATABASE();
I'm trying to get the same results from PostgreSQL. We found a way to do this with a bunch of calls to the database, checking each table individually.
I'd like to make just 1 call to the database. Here's what I have so far:
CREATE OR REPLACE FUNCTION getAllSeqId() RETURNS SETOF record AS
$body$
DECLARE
sequence_name varchar(255);
BEGIN
FOR sequence_name in SELECT relname FROM pg_class WHERE (relkind = 'S')
LOOP
RETURN QUERY EXECUTE 'SELECT last_value FROM ' || sequence_name;
END LOOP;
RETURN;
END
$body$
LANGUAGE 'plpgsql';
SELECT last_value from getAllSeqId() as(last_value bigint);
However, I need to somehow add the sequence_name to each record so that I get output in records of [table_name, last_value] or [sequence_name, last_value].
So I'd like to call my function something like this:
SELECT sequence_name, last_value from getAllSeqId() as(sequence_name varchar(255), last_value bigint);
How can I do this?
EDIT: In ruby, this creates the output we're looking for. As you can see, we're doing 1 call to get all the indexes, then 1 call per index to get the last value. Gotta be a better way.
def perform
find_auto_inc_tables.each do |auto_inc_table|
check_limit(auto_inc_table, find_curr_auto_inc_id(auto_inc_table))
end
end
def find_curr_auto_inc_id(table_name)
ActiveRecord::Base.connection.execute("SELECT last_value FROM #{table_name}").first["last_value"].to_i
end
def find_auto_inc_tables
ActiveRecord::Base.connection.execute(
"SELECT c.relname " +
"FROM pg_class c " +
"WHERE c.relkind = 'S'").map { |i| i["relname"] }
end
Your function seems quite close already. You'd want to modify it a bit to:
include the sequences names as literals
returns a TABLE(...) with typed columns instead of SET OF RECORD because it's easier for the caller
Here's a revised version:
CREATE OR REPLACE FUNCTION getAllSeqId() RETURNS TABLE(seqname text,val bigint) AS
$body$
DECLARE
sequence_name varchar(255);
BEGIN
FOR sequence_name in SELECT relname FROM pg_class WHERE (relkind = 'S')
LOOP
RETURN QUERY EXECUTE 'SELECT ' || quote_literal(sequence_name) || '::text,last_value FROM ' || quote_ident(sequence_name);
END LOOP;
RETURN;
END
$body$
LANGUAGE 'plpgsql';
Note that currval() is not an option since it errors out when the sequence has not been set in the same session (by calling nextval(), not sure if there's any other way).
Would something as simple as this work?
SELECT currval(sequence_name) from information_schema.sequences;
If you have sequences that aren't keys, I guess you could use PG's sequence name generation pattern to try to restrict it.
SELECT currval(sequence_name) from information_schema.sequences
WHERE sequence_name LIKE '%_seq';
If that is still too many false positives, you can get table names from the information_schema (or the pg_* schemata that I don't know very well) and refine the LIKE parameter.
Related
Hello guys ive been trying to build an software using postgresql and python.
Basically i want increment and/or dynamically expand the json
example: at first the field will be empty then:
#insert (toyota,honda,nissan)
{"toyota":1,
"honda":1,
"nissan":1}
#insert (toyota)
{"toyota":2,
"honda":1,
"nissan":1}
#insert (honda,mitsubitshi)
{"toyota":2,
"honda":2,
"nissan":1,
"mitsubitshi":1}
Yes i know it can be done by first retrieving json doing it via python but i dont it that way:
I dont have much experience with postgresql procedure or trigger feature.
Any Help will be apreciated: :-)
Normalized tables would be more performant, however json solution may be quite comfortable using this function:
create or replace function add_cars(cars jsonb, variadic car text[])
returns jsonb language plpgsql as $$
declare
new_car text;
begin
foreach new_car in array car loop
cars = cars || jsonb_build_object(new_car, coalesce(cars->>new_car, '0')::int+ 1);
end loop;
return cars;
end $$;
Find the full example in DbFiddle.
Please check function below. Hopefully it meets your requirement!
CREATE FUNCTION sp_test(json)
RETURNS VOID AS
$BODY$
DECLARE
var_sql varchar;
BEGIN
IF (EXISTS (
SELECT json_object_keys($1)
EXCEPT
SELECT column_name FROM information_schema.columns WHERE table_schema = 'your schema' AND table_name = 'test_table'
)) THEn
RAISE EXCEPTION 'There is column(s) does not exists on table'; -- Checking structure.
END IF;
var_sql := 'Update test_table t SET ' || (SELECT string_agg(CONCAT(t.key, ' = (t.', t.key, ' + ', t.value,')'),', ') FROM json_each($1) t);
EXECUTE (var_sql);
END;
$BODY$
LANGUAGE plpgsql;
I have a Mysql table name xyz which has two columns tc_name and tc_path where tc_path is like /a/b/c/x/tc_name and tc_name is null .I want to insert the values in the column(tc_name) using this tc_path column values.
I know it would be something like we do with regular expressions in other languages.But i have no idea how to do this in Mysql.
I source the function:
DELIMITER $$
CREATE FUNCTION `regex_replace`(pattern VARCHAR(1000),replacement VARCHAR(1000),original VARCHAR(1000))
RETURNS VARCHAR(1000)
DETERMINISTIC
BEGIN
DECLARE temp VARCHAR(1000);
DECLARE ch VARCHAR(1);
DECLARE i INT;
SET i = 1;
SET temp = '';
IF original REGEXP pattern THEN
loop_label: LOOP
IF i>CHAR_LENGTH(original) THEN
LEAVE loop_label;
END IF;
SET ch = SUBSTRING(original,i,1);
IF NOT ch REGEXP pattern THEN
SET temp = CONCAT(temp,ch);
ELSE
SET temp = CONCAT(temp,replacement);
END IF;
SET i=i+1;
END LOOP;
ELSE
SET temp = original;
END IF;
RETURN temp;
END$$
I tried this:
update testplans set tc_name=regex_replace('.*/','',tc_path);
It should have given me tc_name=d if tc_path=/a/b/c/d
But instead it gave me abcd.
Am i using a wrong function or i don't know how to use it.
Please suggest some query for it.
Please help!!!
Thanks
In THIS particular case, where you are trying to get the last part of a path, you can skip regex and use simple string functions:
update testplans set tc_name=substr(tc_path, length(tc_path)-locate('/',reverse(tc_path))+2);
Doing search in reverse order
The proper regex expression would be
.*/([^/]*)
But you'd need to extract the last part, which MySQL really doesn't do. MariaDB 10 has a REGEXP_REPLACE that might help.
The function you have above will replace single characters that match a regex, but it's not a full regular expression replacement. Basically, wrong function.
Mysql isn't great at this right now -- you might want to consider another path, like the one that Jacques suggested, with a simple string replace.
If you really want a regex solution, maybe look at a more complete regex library for mysql:
https://github.com/mysqludf/lib_mysqludf_preg
I wrote a function to generate unique id's,its working but sometimes two people are getting same id,I mean duplicates are formed. My unique id looks like
2016-17NLR250001, I deal with only last four digits 0001. I am posting my function please correct it and please help me in avoiding duplicates even though users login into same account or if they do it on same time.
MY FUNCTION:
DELIMITER $$
USE `olmsap`$$
DROP FUNCTION IF EXISTS `fun_generate_uniqueid`$$
CREATE DEFINER=`root`#`%` FUNCTION `fun_generate_uniqueid`( V_DATE DATE,V_MANDALID INT ) RETURNS VARCHAR(30) CHARSET latin1
DETERMINISTIC
BEGIN
DECLARE MDLCODE VARCHAR(5);
SET MDLCODE = ' ';
SELECT COUNT(*) INTO #CNT FROM `st_com_mandal` WHERE MANDAL_VS_MC=V_MANDALID;
SELECT dist_mandal_code INTO MDLCODE FROM `st_com_mandal` WHERE MANDAL_VS_MC=V_MANDALID;
IF #CNT>0 THEN
SET #YR=`FUN_FISCAL_YR`(V_DATE);
SELECT CONCAT(IF(DIST_SAN_CODE='GUN','GNT',DIST_SAN_CODE),IFNULL(`dist_mandal_code`,'NULL'))INTO #MANDAL
FROM `st_com_dist` SCD INNER JOIN `st_com_mandal` STM ON STM.`mandal_dist_id`= SCD.`DIST_VC_DC` WHERE MANDAL_VS_MC=V_MANDALID;
IF MDLCODE >0 THEN
SELECT COUNT(Soil_Sample_ID)+1 INTO #ID FROM `tt_mao_soil_sample_dtls` WHERE MANDAL_ID=V_MANDALID AND SUBSTR(UNIQUE_ID,1,7)=#YR ;
ELSE
SELECT COUNT(Soil_Sample_ID)+1 INTO #ID FROM `tt_mao_soil_sample_dtls` WHERE SUBSTR(UNIQUE_ID,1,14)=CONCAT(#YR,#MANDAL) ;
END IF ;
IF LENGTH(#ID)=1 THEN
SET #ID=CONCAT('000',#ID);
ELSEIF LENGTH(#ID)=2 THEN
SET #ID=CONCAT('00',#ID);
ELSEIF LENGTH(#ID)=3 THEN
SET #ID=CONCAT('0',#ID);
ELSE
SET #ID=#ID;
END IF ;
RETURN CONCAT(#YR,#MANDAL,#ID);
ELSE
RETURN 'Mandal Doesnt Exists';
END IF;
END$$
DELIMITER ;
I do not think community will be able to help you with this question. This is a complex function that requires very careful analysis of table / index access and locking.
The only thing I can recommend is to not use existing table data to calculate next sequence as this is a bad practice.
Besides Race conditions that you are experiencing you will also get problems if the record with the last sequence is deleted.
I suggest you read this to get an idea on how to write a custom sequence generator:
http://en.latindevelopers.com/ivancp/2012/custom-auto-increment-values/
Can we select inside a PostgreSQL function without using EXECUTE?
I'm trying to use quote_ident() to create dynamic SQL but it doesn't work.
CREATE OR REPLACE FUNCTION select_server(p_id text)
RETURNS integer AS $$
DECLARE
serialnum_value INTEGER;
STATEMENT TEXT;
BEGIN
STATEMENT := 'tbl' || substr($1, 1, 4);
SELECT serialnum INTO serialnum_value FROM quote_ident(STATEMENT ) WHERE id = $1;
RETURN serialnum;
END;
Does anybody have idea how to select from dynamic table in a PostgreSQL function without using EXECUTE?
You can't avoid execute here, per igor's comment. The best alternative is to create a function that creates a function, which you can then call as needed. Example:
create function gen_select_server(p_id text)
returns boolean
as $gen$
begin
execute $exec$
create function $exec$ || quote_ident('select_server_' || p_id) || $exec$
returns integer
as $body$
begin
return serialnum
from $exec$ || quote_ident('tbl_' || p_id) || $exec$
where id = $exec$ || quote_literal('id_' || p_id) || $exec$;
end;
$body$ language plpgsql;
$exec$;
return true;
end;
$gen$ language plpgsql;
-- usage:
select gen_select_server('1234'); -- call this only once; uses execute
select * from select_server_1234(); -- no execute here
As the above highlights, it can get messy with string delimiters. Also note that the above isn't your original function -- it's primarily to illustrate how to quote things properly within the big $exec$ "string" block.
I'd recommend to stick to using execute, that being said. Or using an ORM, for that matter. Maintaining these micro-optimizations is potentially a pain worse than the superficial gain in performance.
You cannot execute dynamic SQL without EXECUTE in PL/pgSQL. That's what makes it dynamic in the first place.
But you can streamline your function quite a bit:
CREATE OR REPLACE FUNCTION select_server(p_id text, OUT serial_value integer) AS
$func$
BEGIN
EXECUTE 'SELECT serialnum FROM tbl' || left($1::text, 4)) || ' WHERE id = $1'
USING $1
INTO serial_value;
END
$func$ LANGUAGE plpgsql STRICT;
Normally you would have to sanitize any identifier.
But you do not need quote_ident() in this special case, since the only dynamic component are 4 digits from an integer. Neither SQL injection nor illegal identifiers are possible this way.
Pass the value of p_id as value using the USING clause.
Reduce the number of assignments. Those are comparatively expensive in plpgsql. You only need a single SQL statement to do everything.
The OUT parameter helps to shorten the syntax.
I also made the function STRICT (RETURNS NULL ON NULL INPUT), since it would not make sense with NULL as input.
left() is slightly faster than substr().
I am trying to have a conditional change in a parameter for update statement.
I am getting the following error when I try the following function
/home/y/bin/mysql -u root < testpri.sql > out
ERROR 1415 (0A000) at line 4: Not allowed to return a result set from a function
Contents of testpri.sql are as follows:
use `zestdb`;
DROP FUNCTION IF EXISTS UPDATEPASSWD;
DELIMITER //
CREATE FUNCTION UPDATEPASSWD(n INT) RETURNS varchar(255) DETERMINISTIC
BEGIN
DECLARE mypasswd varchar(255);
IF (n = 1) THEN
SET mypasswd = '12ccc1e5c3c9203af7752f937fca4ea6263f07a5';
SELECT 'n is 1' AS ' ';
ELSE
SET mypasswd = '1a7bc371cc108075cf8115918547c3019bf97e5d';
SELECT 'n is 0' AS ' ';
END IF;>
SELECT CONCAT('mypasswd is ', mypasswd) AS ' ';
RETURN mypasswd;
END //
DELIMITER ;
CALL UPDATEPASSWD(0);
What am I missing?
I think it's actually your debugging SELECT calls.
From the docs:
Statements that return a result set can be used within a stored procedure but not within a stored function. This prohibition includes SELECT statements that do not have an INTO var_list clause...
I arrived in search of answers to the same question, and found another way to work around the issue, so that I can use the SELECT statement that is the heart and soul of the MySQL function that elicited the warning.
Consider the following snippet.
SET intNMatches = ( SELECT COUNT(*) ...
SET coerces the SELECT statement to return its one and only column, a row count, into intNMatches, a local variable cast to BIGINT. Since it contains trade secrets, I can't show the rest of the query. Suffice it to say that the query installs without causing the MySQL engine to issue a warning.