What I'd like to do is execute a MySQL query containing a where clause ("result query") that is stored in a column in the database. This column, containing the query, is a result of another query ("original query").
The catches:
The result query's where clause can contain a variable value (or two)
I don't know what the result query will be when executing the original query, so I cannot pass along the variable's value
(The list of result queries as well as the variables in the where clauses will be defined by me, so I will will have a list of all the possible variables.)
Essentially, I need to be able to correlate that variable with any number of other values, one example: a user_id, within the database.
original_query_table | result_query_table
--------------------------------------------------------------
other_id result_query_id | result_query_id result_query
1 1 1 "SELECT ... WHERE user_id = :id "
I know how to do this with two separate queries, but the question is whether this is possible with only one query?
I would do something like this:
SELECT 'select * from table_a where col_a = ?' INTO #query, 1 into #param1 FROM dual;
PREPARE stmt FROM #query;
EXECUTE stmt USING #param1 ;
So converting that into your tables, I guess would look like:
SELECT a.result_query INTO #query, b.result_query_id INTO #param1 FROM result_query_table a, original_query_table b where a.result_query_id = b.result_query_id;
PREPARE stmt FROM #query;
EXECUTE stmt USING #param1 ;
Will you know how many parameters the where clause will need? If that's dynamic, then things could get a bit tricky.
Related
I have a scenario where the result must be limited depends on counting ids in another table.
Suppose i have these two tables counter and dispanser,
i want to select the last records in the table counter and limit the selection by counting the number of records in dispanser table.
something like this
select * from counter limit (select count(dispID) from dispanser)
You can't do this without using prepared statements or a stored procedure. From the manual:
LIMIT takes one or two numeric arguments, which must both be nonnegative integer constants
In a stored procedure you could do something like this. COUNT(dispID) is stored into cnt and that variable is then used as the parameter to LIMIT. This is the exception to the above-mentioned rule.
DELIMITER //
CREATE PROCEDURE select_counter()
BEGIN
DECLARE cnt INT;
SELECT COUNT(dispID) INTO cnt FROM dispanser;
SELECT * FROM counter LIMIT cnt;
END //
DELIMITER ;
DBFiddle
Based on conversations in the comments, it sounds like what you're trying to do is get the count from counter for each record in dispanser - if this is wrong, please comment, and I can adjust my response. The best way to accomplish what you're looking for is through joining a subquery with the GROUP BY syntax. Something like this might could work, depending on your schema:
SELECT
d.*,
c.total
FROM
dispanser as d
INNER JOIN (
SELECT
COUNT(*) as 'total',
dispID
FROM
counter
GROUP BY
dispID
) as c
ON c.dispID = d.id
You can try to use dynamic SQL.
Set a variable #num to get an amount from dispanser table.
prepared your SQL statement CONCAT('select * from counter limit ', #num ).
Final use EXECUTE to execute SQL dynamically.
SET #sql = CONCAT('select * from counter order by counterID desc limit ', (SELECT count(dispID) from dispanser));
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
sqlfiddle
I know this is a bad example but I'm trying to simplify things so bear with me in the round-about way this code is written. Say I have queries stored in a database such as
queries table
id query
1 concat('SELECT * FROM table1 WHERE year = ', _year, 'order by name')
2 concat('SELECT * FROM table2 WHERE year = ', _year, 'order by name')
and I want to run the following routine
DECLARE _year;
SET _year= "2013";
SET #SQL = (SELECT query FROM queries WHERE id = 1);
PREPARE stmt FROM #SQL;
EXECUTE stmt;
this is what I currently have but it's not working. I'm trying to select a query from the database, pass a few variables into it and then run the query.
If it is possible to determine your query in advance, except for specific parameters, then you could consider using a UNION query with a discriminant. Then, use your queries table to select specific queries in the union and apply parameters to them. The queries themselves are defined in a view in the database.
SQL is intended in most DBMS to not be dynamic and the attempt to subvert this will almost certainly result in reduced performance and potential security problems.
CREATE VIEW UnionView
AS SELECT *, 1 as Type, Value1 AS Param FROM Table1
UNION ALL SELECT *, 2 as Type, Value2 AS Param FROM Table1
UNION ALL SELECT *, 3 as Type, Value3 AS Param FROM Table1;
SELECT * FROM UnionView WHERE Type = 2 AND Param = 2;
See SqlFiddle for an example to demonstrate the behaviour.
My ultimate goal is to create a pivot-table view in MySQL with dynamic columns based on the contents of another table. At the moment I am trying to continue on from where artfulsoftware leaves off; right now I can query for results that give me my desired column names. Unfortunately I'm lost on how to actually use the results as column names in a SELECT statement. I suspect that MySQL variables will be helpful, but I can't figure it out.
To clarify the problem, say I have a table like:
+---------------------------------------------------+
| countpivotarg |
+---------------------------------------------------+
| ,SUM(IF(domain = "test.com",1,0)) AS `test.com` |
| ,SUM(IF(domain = "test2.com",1,0)) AS `test2.com` |
+---------------------------------------------------+
I want to create a select statement that looks like:
SELECT id,
meta_id,
SUM(IF(domain = "test.com",1,0)) AS `test.com`,
SUM(IF(domain = "test2.com",1,0)) AS `test2.com`
FROM myTable;
How do I go about doing that?
In SQL, the column names must be fixed at query prepare time -- no exceptions. Designing a dynamic pivot query when you don't know the columns requires that you write application code either way. You have a choice between using a preprocessing approach or a postprocessing approach:
Preprocessing: Write a query to fetch a list of the distinct values. Then use these as column names, and generate a new dynamic SQL query.
Postprocessing: Write a query to fetch the data as a non-pivoted result, and then fetch all the data and pivot into another format.
You may use MySQL Server prepared statements for building dynamic queries from string variables.
Example:
SELECT domain INTO #colname FROM myTable LIMIT 1;
SET #s = CONCAT('SELECT `',#colname,'` FROM myTable');
PREPARE stmt FROM #s;
EXECUTE stmt;
Note: backticks are required in case column names contain spaces.
I have a query that picks out a specific row/column from a large database. Lets say that the value returned is '53.'
I need to get:
1. The row that is 3000 rows above this value.
2. The row that is 3000 rows below this value.
If there turn out to be only 2000 rows above the value, then I need to add the difference onto the second query.
Ex.
1. Find 3000th value up (turns out that only 2000 values are present)
2. Find 4000th value down.
Here is how I did it (this is in a stored procedure):
SET #s1 = CONCAT('INSERT INTO List1(STD) SELECT t1.STD FROM ',t1,' AS t1 USE INDEX(STD) WHERE t1.STD < ',inp1,' order by STD DESC limit ', inp2);
PREPARE stmt FROM #s1;
EXECUTE stmt;
SET #lim = inp2+(inp2-(SELECT FOUND_ROWS()));
SET #s2 = CONCAT('INSERT INTO List1(STD) SELECT t1.STD FROM ',t1,' AS t1 USE INDEX(STD) WHERE t1.STD >=', inp1,' order by STD ASC limit ?');
PREPARE stmt FROM #s2;
EXECUTE stmt USING #lim;
SET #minSD1 = (SELECT MIN(STD) FROM List1);
SET #maxSD1 = (SELECT MAX(STD) FROM List1);
This seems awfully round-about... is there no better way?
Also, is there really no way to use a variable table name in a stored procedure without creating ANOTHER stored procedure (with the PREPARE keyword)?
In SQL, the concept of "3000 rows above/below a given point" exists ONLY in the context of an ordered resultset, and is not defined in the database. Unless there's an algorithmic process, based on a key, to determine what is "n rows above/below" a given point, then you are stuck with actually reading the rows and counting, which is what your solution seems to attempt. I didn't verify that it actually does what you want, but either this approach or one based on cursors and counting rows is the only way to get this done.
Just wondering, is there any quick way to count all the NULL values (from all columns) in a MySQL table?
Thanks for any idea!
If you want this done exclusively by MYSQL and without enumerating all of the columns take a look at this solution.
In this method you don't have to maintain the number of database columns by hard coding them. If your table schema will get modified this method will work, and won't require code change.
SET #db = 'testing'; -- database
SET #tb = 'fuzzysearch'; -- table
SET #x = ''; -- will hold the column names with ASCII method applied to retrieve the number of the first char
SET #numcolumns = 0; -- will hold the number of columns in the table
-- figure out how many columns we have
SELECT count(*) into #numcolumns FROM information_schema.columns where table_name=#tb and table_schema=#db;
-- we have to prepare some query from all columns of the table
SELECT group_concat(CONCAT('ASCII(',column_name,')') SEPARATOR ",") into #x from information_schema.columns where table_name=#tb and table_schema=#db;
-- after this query we have a variable separated with comma like
-- ASCII(col1),ASCII(col2),ASCII(col3)
-- we now generate a query to concat the columns using comma as separator (null values are omitted from concat)
-- then figgure out how many times the comma is in that substring (this is done by using length(value)-length(replace(value,',',''))
-- the number returned is how many non null columns we have in that column
-- then we deduct the number from the known number of columns, calculated previously
-- the +1 is added because there is no comma for single value
SET #s = CONCAT('SELECT #numcolumns - (length(CONCAT_WS(\',\',', #x, '))-length(replace(CONCAT_WS(\',\',', #x, '),\',\',\'\')) + 1) FROM ',#db,'.',#tb,';');
PREPARE stmt FROM #s;
EXECUTE stmt;
-- after this execution we have returned for each row the number of null columns
-- I will leave to you to add a sum() group call if you want to find the null values for the whole table
DEALLOCATE PREPARE stmt;
The ASCII is used to avoid reading, concatenating very long columns for nothing, also ASCII makes us safe for values where the first char is a comma(,).
Since you are working with reports, you may find this helpful as this can be reused for each table if you put in a method.
I tried to let as many comments as possible.
Let's split on pieces the above compact way (reverse way):
I wanted to end up having a query like this
SELECT totalcolumns - notnullcolumns from table; -- to return null columns for each row
While the first one is easy to calcule by running:
SELECT count(*) FROM information_schema.columns where table_name=#tb and table_schema=#db;
The second one the notnullcolumns is a bit of pain.
After a piece of examination of the functions available in MySQL, we detect that CONCAT_WS does not CONCAT null values
So running a query like this:
SELECT CONCAT_WS(",","First name",NULL,"Last Name");
returns: 'First name,Last Name'
This is good, we take rid of the null values from the enumeration.
But how do we get how many columns were actually concatenated?
Well that is tricky. We have to calculate the number of commas+1 to get the actually concatenated columns.
For this trick we used the following SQL notation
select length(value)-length(replace(value,',','')) +1 from table
Ok, so we have now the number of concatenated columns.
But the harder part is coming next.
We have to enumerate for CONCAT_WS() all values.
We need to have something like this:
SELECT CONCAT_WS(",",col1,col2,col3,col4,col5);
This is where we have to take use of the prepared statements, as we have to prepare an SQL query dynamically from yet unknown columns. We don't know how many columns will be in our table.
So for this we use data from information_schema columns table. We need to pass the table name, but also the database name, as we might have the same table name in separate databases.
We need a query that returns col1,col2,col3,col4,col5 to us on the CONCAT_WS "string"
So for this we run a query
SELECT group_concat(column_name SEPARATOR ",") into #x from information_schema.columns where table_name=#tb and table_schema=#db;
One more thing to mention. When we used the length() and replace() method to find out how many columns were concatenated, we have to make sure we do not have commas among the values. But also take note that we can have really long values in our database cells. For both of this trick we use method ASCII('value'), which will return the ASCII char of the first char, which cannot be comma and will return null for null columns.
That being said we can compact all this in the above comprehensive solution.
Something like
select id
, sum ( case when col1 is null then 1 else 0 end case ) col1
, sum ( case when col2 is null then 1 else 0 end case ) col2
, sum ( case when col3 is null then 1 else 0 end case ) col3
from contacts
group by id
Something like this (substitute COL_COUNT as appropriate):
select count(*) * COL_COUNT - count(col1) - count(col2) - ... - count(col_n) from table;
You should really do this using not only SQL, but the language which is at your disposal:
Obtain the metadata of each table - either using DESCRIBE table, or using a built-in metadata functionality in your db access technology
Create queries of the following type in a loop for each column. (in pseudo-code)
int nulls = 0;
for (String colmnName : columNames) {
query = "SELECT COUNT(*) FROM tableName WHERE " + columnName + " IS NULL";
Result result = executeQuery(query);
nulls += result.size();
}