Note - I have edited to better reflect the complexity on the math. Sorry, very new to this.
I am trying to repeatedly run a mathematical function on certain metrics in a dataset in MySQL. The mathematical lag function involves several variables but I only need to alter one of them. The idea is to create this as a function and then call the function in a view. I am able to create the function and the view. However, when I call the view I receive
Error Code: 1366. Incorrect decimal value: 'Variable' for column ``.``.`metric` at row 280
Normally this would just indicate bad data entry from the original table. But here's the catch - the table only has 279 observations. Why is my function trying to interpret nonexistent rows?
I've tried pulling out just the select function and it works fine so it's not a problem there.
Here is the code I've used with some dummy names to preserve data integrity. The actual formula is more complicated but the same error occurs with even basic math so I've used that.
To create the function
DELIMITER //
DROP FUNCTION IF EXISTS Levels;
CREATE FUNCTION Levels(
metric decimal(65,4),
dataset VARCHAR(60)
)
RETURNS DECIMAL(65,4)
BEGIN
DECLARE levels DECIMAL(65,4);
SET levels = NULL;
SELECT
metric - (LAG(metric,1) OVER (PARTITION BY clown_id ORDER BY year));
INTO levels
FROM dataset;
RETURN levels;
END //
To create and call view
DROP VIEW IF EXISTS Circus_Mainfile;
CREATE VIEW Circus_Mainfile AS
SELECT
my_database.circus_inputs.*,Levels('Clowns', 'my_database.circus_inputs`) AS More_clowns
FROM my_database.circus_inputs;
SELECT * FROM Circus_Mainfile LIMIT 279;
Thanks in advance.
The function shouldn't select from the table, that's done by the view itself. The function should just receive the value that was selected, and perform the calculation.
CREATE FUNCTION Levels(prev_level DECIMAL(65, 4)) RETURNS DECIMAL(65, 4)
BEGIN
RETURN prev_level + 2
END
Then you call it in the view like this:
CREATE VIEW Circus_Mainfile AS
SELECT i.*,
Levels(i.clowns) AS More_clowns
FROM my_database.circus_inputs AS i;
I have a piece of code (select and drop tables) where i use functions as well.
But if run function with other codes, it says:-
CREATE FUNCTION MUST BE THE ONLY STATEMENT IN THE BATCH
So can i only run the function code once and then remove it from the weekly job code (the creating and dropping of tables) ?
Or do i do something else.
This is one of the functions iam using:-
CREATE FUNCTION dbo.getyrs(#parm Integer)
RETURNS INTEGER
AS
BEGIN
RETURN #parm
END;
But if i use drop function after this it gives an error (i can't run both drop and create functions together like i can run create and drop tables)
Put GO before the CREATE FUNCTION statement.
Also put it before any other statement that creates an object.
The GO statement signifies the start of a new Batch so any time you get an error about how a certain statement needs to be the first statement in a batch, or the only statement in a batch, you can use the GO statement to separate the code into batches.
This code works for me, no errors:
Go
CREATE FUNCTION dbo.getyrs(#parm Integer)
RETURNS INTEGER
AS
BEGIN
RETURN #parm
END;
GO
DROP FUNCTION dbo.getyrs;
Why would you need to keep creating and dropping the function. Normally =you would create it once, and then call it for example like
SELECT schema.functionname(#paramatername)
Or you reference it in an outer apply or a create table etc
This question already has answers here:
How can I get the number of rows 'returned' from a result set of a stored procedure
(2 answers)
Closed 7 years ago.
I am trying to write a function to return the number of rows a call to a stored procedure would return. I'm trying to minimise repetition of the code (for reduced code maintenance/debugging- the procedure select is long).
The stored procedure just read-only selects rows matching certain criteria (vague I know but details should not be material to the question).
I could just copy the procedure into a function and change the select to count() but as it is long with multiple joins I was hoping to write a function that could call the procedure and return the row count. The goal is not for optimised running but for efficient code maintenance, boiler plate reduction.
I have tried this as a test:
DELIMITER //
CREATE PROCEDURE IF NOT EXISTS proc_select1()
BEGIN
SELECT 1;
END //
CREATE FUNCTION IF NOT EXISTS select1_count() RETURNS INT UNSIGNED
BEGIN
CALL proc_select1();
RETURN FOUND_ROWS();
END //
DELIMITER ;
However when I SELECT select1_count(); - which I am hoping will return 1 - I get the "cannot return a result set from a function" error.
I tried assigning FOUND_ROWS to a variable, clearing the result set then returning the variable value but can't get it to work.
Does anyone know a work around or do I really need to copy-paste the procedure and convert to a SELECT COUNT and function?
I'm using MySQL 5.5.16 (can upgrade if necessary), Windows 7 (nobody seems to want to upgrade :) with HeidiSQLv7.0.0.4053 (if relevant)
As always, any help much appreciated.
first use distinct to get distinct values then use count on that..like
select count(distinct column_name) from table_name
cannot return a result set from a function
This error happens when a SELECT query is done in a procedure without storing the output values.
Other thing : don't forget to use SQL_CALC_FOUND_ROWS to indicates your DBMS to store the number of found rows.
I tried to make it work without any temporary variable (Maybe there is exists a mysql keyword to not "return" the set of the SELECT query), but no success. Here a piece of code which works using temporary var.
CREATE PROCEDURE proc_select1()
BEGIN
DECLARE temp INT;
SELECT SQL_CALC_FOUND_ROWS 1 INTO temp;
END //
CREATE FUNCTION select1_count() RETURNS INT UNSIGNED
BEGIN
CALL proc_select1();
RETURN FOUND_ROWS();
END //
The result is 1 as expected :-) cf : SQLFiddle
It appears this is not possible. The select statement is required to SELECT INTO something to avoid the cannot return a result set from a function error during the function call. If this is not possible or required in the SELECT statement used in the procedure then the function will not run without error.
Using CLEAR QUERY CACHE or FLUSH QUERY CACHE after the procedure call did not help (and is probably a bad idea / bad coding anyway).
I want to pass a table name as a parameter in a Postgres function. I tried this code:
CREATE OR REPLACE FUNCTION some_f(param character varying) RETURNS integer
AS $$
BEGIN
IF EXISTS (select * from quote_ident($1) where quote_ident($1).id=1) THEN
return 1;
END IF;
return 0;
END;
$$ LANGUAGE plpgsql;
select some_f('table_name');
And I got this:
ERROR: syntax error at or near "."
LINE 4: ...elect * from quote_ident($1) where quote_ident($1).id=1)...
^
********** Error **********
ERROR: syntax error at or near "."
And here is the error I got when changed to this select * from quote_ident($1) tab where tab.id=1:
ERROR: column tab.id does not exist
LINE 1: ...T EXISTS (select * from quote_ident($1) tab where tab.id...
Probably, quote_ident($1) works, because without the where quote_ident($1).id=1 part I get 1, which means something is selected. Why may the first quote_ident($1) work and the second one not at the same time? And how could this be solved?
Before you go there: for only few, known tables names, it's typically simpler to avoid dynamic SQL and spell out the few code variants in separate functions or in a CASE construct.
That said, what you are trying to achieve can be simplified and improved:
CREATE OR REPLACE FUNCTION some_f(_tbl regclass, OUT result integer)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format('SELECT (EXISTS (SELECT FROM %s WHERE id = 1))::int', _tbl)
INTO result;
END
$func$;
Call with schema-qualified name (see below):
SELECT some_f('myschema.mytable'); -- would fail with quote_ident()
Or:
SELECT some_f('"my very uncommon table name"');
Major points
Use an OUT parameter to simplify the function. You can directly select the result of the dynamic SQL into it and be done. No need for additional variables and code.
EXISTS does exactly what you want. You get true if the row exists or false otherwise. There are various ways to do this, EXISTS is typically most efficient.
You seem to want an integer back, so I cast the boolean result from EXISTS to integer, which yields exactly what you had. I would return boolean instead.
I use the object identifier type regclass as input type for _tbl. That does everything quote_ident(_tbl) or format('%I', _tbl) would do, but better, because:
.. it prevents SQL injection just as well.
.. it fails immediately and more gracefully if the table name is invalid / does not exist / is invisible to the current user. (A regclass parameter is only applicable for existing tables.)
.. it works with schema-qualified table names, where a plain quote_ident(_tbl) or format(%I) would fail because they cannot resolve the ambiguity. You would have to pass and escape schema and table names separately.
It only works for existing tables, obviously.
I still use format(), because it simplifies the syntax (and to demonstrate how it's used), but with %s instead of %I. Typically, queries are more complex so format() helps more. For the simple example we could as well just concatenate:
EXECUTE 'SELECT (EXISTS (SELECT FROM ' || _tbl || ' WHERE id = 1))::int'
No need to table-qualify the id column while there is only a single table in the FROM list. No ambiguity possible in this example. (Dynamic) SQL commands inside EXECUTE have a separate scope, function variables or parameters are not visible there - as opposed to plain SQL commands in the function body.
Here's why you always escape user input for dynamic SQL properly:
db<>fiddle here demonstrating SQL injection
Old sqlfiddle
If at all possible, don't do this.
That's the answer—it's an anti-pattern. If the client knows the table it wants data from, then SELECT FROM ThatTable. If a database is designed in a way that this is required, it seems to be designed sub-optimally. If a data access layer needs to know whether a value exists in a table, it is easy to compose SQL in that code, and pushing this code into the database is not good.
To me this seems like installing a device inside an elevator where one can type in the number of the desired floor. After the Go button is pressed, it moves a mechanical hand over to the correct button for the desired floor and presses it. This introduces many potential issues.
Please note: there is no intention of mockery, here. My silly elevator example was *the very best device I could imagine* for succinctly pointing out issues with this technique. It adds a useless layer of indirection, moving table name choice from a caller space (using a robust and well-understood DSL, SQL) into a hybrid using obscure/bizarre server-side SQL code.
Such responsibility-splitting through movement of query construction logic into dynamic SQL makes the code harder to understand. It violates a standard and reliable convention (how a SQL query chooses what to select) in the name of custom code fraught with potential for error.
Here are detailed points on some of the potential problems with this approach:
Dynamic SQL offers the possibility of SQL injection that is hard to recognize in the front end code or the back end code alone (one must inspect them together to see this).
Stored procedures and functions can access resources that the SP/function owner has rights to but the caller doesn't. As far as I understand, without special care, then by default when you use code that produces dynamic SQL and runs it, the database executes the dynamic SQL under the rights of the caller. This means you either won't be able to use privileged objects at all, or you have to open them up to all clients, increasing the surface area of potential attack to privileged data. Setting the SP/function at creation time to always run as a particular user (in SQL Server, EXECUTE AS) may solve that problem, but makes things more complicated. This exacerbates the risk of SQL injection mentioned in the previous point, by making the dynamic SQL a very enticing attack vector.
When a developer must understand what the application code is doing in order to modify it or fix a bug, he'll find it very difficult to get the exact SQL query being executed. SQL profiler can be used, but this takes special privileges and can have negative performance effects on production systems. The executed query can be logged by the SP but this increases complexity for questionable benefit (requiring accommodating new tables, purging old data, etc.) and is quite non-obvious. In fact, some applications are architected such that the developer does not have database credentials, so it becomes almost impossible for him to actually see the query being submitted.
When an error occurs, such as when you try to select a table that doesn't exist, you'll get a message along the lines of "invalid object name" from the database. That will happen exactly the same whether you're composing the SQL in the back end or the database, but the difference is, some poor developer who's trying to troubleshoot the system has to spelunk one level deeper into yet another cave below the one where the problem exists, to dig into the wonder-procedure that Does It All to try to figure out what the problem is. Logs won't show "Error in GetWidget", it will show "Error in OneProcedureToRuleThemAllRunner". This abstraction will generally make a system worse.
An example in pseudo-C# of switching table names based on a parameter:
string sql = $"SELECT * FROM {EscapeSqlIdentifier(tableName)};"
results = connection.Execute(sql);
While this does not eliminate every possible issue imaginable, the flaws I outlined with the other technique are absent from this example.
Inside plpgsql code, The EXECUTE statement must be used for queries in which table names or columns come from variables. Also the IF EXISTS (<query>) construct is not allowed when query is dynamically generated.
Here's your function with both problems fixed:
CREATE OR REPLACE FUNCTION some_f(param character varying) RETURNS integer
AS $$
DECLARE
v int;
BEGIN
EXECUTE 'select 1 FROM ' || quote_ident(param) || ' WHERE '
|| quote_ident(param) || '.id = 1' INTO v;
IF v THEN return 1; ELSE return 0; END IF;
END;
$$ LANGUAGE plpgsql;
I know this is an old thread, but I ran across it recently when trying to solve the same problem - in my case, for some fairly complex scripts.
Turning the entire script into dynamic SQL is not ideal. It's tedious and error-prone work, and you lose the ability to parameterize: parameters must be interpolated into constants in the SQL, with bad consequences for performance and security.
Here's a simple trick that lets you keep the SQL intact if you only need to select from your table - use dynamic SQL to create a temporary view:
CREATE OR REPLACE FUNCTION some_f(_tbl varchar) returns integer
AS $$
BEGIN
drop view if exists myview;
execute format('create temporary view myview as select * from %s', _tbl);
-- now you can reference myview in the SQL
IF EXISTS (select * from myview where myview.id=1) THEN
return 1;
END IF;
return 0;
END;
$$ language plpgsql;
The first doesn't actually "work" in the sense that you mean, it works only in so far as it does not generate an error.
Try SELECT * FROM quote_ident('table_that_does_not_exist');, and you will see why your function returns 1: the select is returning a table with one column (named quote_ident) with one row (the variable $1 or in this particular case table_that_does_not_exist).
What you want to do will require dynamic SQL, which is actually the place that the quote_* functions are meant to be used.
If the question was to test if the table is empty or not (id=1), here is a simplified version of Erwin's stored proc :
CREATE OR REPLACE FUNCTION isEmpty(tableName text, OUT zeroIfEmpty integer) AS
$func$
BEGIN
EXECUTE format('SELECT COALESCE ((SELECT 1 FROM %s LIMIT 1),0)', tableName)
INTO zeroIfEmpty;
END
$func$ LANGUAGE plpgsql;
If you want table name, column name and value to be dynamically passed to function as parameter
use this code
create or replace function total_rows(tbl_name text, column_name text, value int)
returns integer as $total$
declare
total integer;
begin
EXECUTE format('select count(*) from %s WHERE %s = %s', tbl_name, column_name, value) INTO total;
return total;
end;
$total$ language plpgsql;
postgres=# select total_rows('tbl_name','column_name',2); --2 is the value
I have 9.4 version of PostgreSQL and I always use this code:
CREATE FUNCTION add_new_table(text) RETURNS void AS
$BODY$
begin
execute
'CREATE TABLE ' || $1 || '(
item_1 type,
item_2 type
)';
end;
$BODY$
LANGUAGE plpgsql
And then:
SELECT add_new_table('my_table_name');
It works good for me.
Attention! Above example is one of those which shows "How do not if we want to keep safety during querying the database" :P
I am trying to use the result of a stored function in a WHERE statement in MySQL (5.x), but it fails because in the function I am selecting values from a table into an INT variable and then returning them/it, which obviously doesn't work if the SELECT returns more than 1 row. I've tried returning a TABLE (as I understood TABLE means array in MySQL) but that didn't work either.
Is there any way that I could do something like:
SELECT ID FROM myTable WHERE ID IN my_function(params);
Thank you.
This cannot be done...
First, you cannot use a stored function to return multiple results - you would need to use a stored procedure.
The MySQL docs state:
Statements that return a result set can be used within a stored procedure but not within a stored function.
Second, you cannot use a stored procedure in a query - see this SO question.
Have you considered using 'HAVING ...' at the end of your query?