Complex WHERE clause in MySQL query - [Use VLOOKUP in MySQL] - mysql

I have a database with following tables (with sample data in parenthesis):
companies {
id: (1, 2, 3)
company_name: ('Goog', 'Micr', 'Apple')
}
companies_ratios {
company_id: (1, 1, 2, 1, 1)
ratio_id: (1, 2, 1, 4, 5)
value: (13, 9, 15, 5, 6)
}
ratios {
id: (1, 2, 3, 4, 5)
ratio_name: ('CAGR', 'Prf. Gwt', 'Sal. Gwt', 'Sales_2012', 'Sales_2011')
ratio_formula: ('...')
-- ratio_formula is not used at the moment
}
I need to parse user query into MySQL to return matching companies. However, I can't find a solution to parse some simple queries.
Eg. User Queries:
Sales_2012 > 1.1 * Sales_2012 AND
CAGR > 13
Prf. Gwt > Sal. Gwt OR
CAGR > Prf. Gwt
Main Problem in parsing above:
I plan to replace all ratio names in user query with their value in companies_ratios. However, I need to cross-reference row data. I need something like value where ratio_id = x for each ratio_name. It might be similar to vlookup in excel (to use row data as column).

I have been able to create a stored procedure based on the structure you have provided that returns the company id from a string. It is a little complex, and may need others to review it, but it appears to provide the correct results.
DELIMITER $$
DROP PROCEDURE IF EXISTS sp_build_query $$
CREATE PROCEDURE sp_build_query(IN userquery VARCHAR(100))
BEGIN
DECLARE no_more_rows BOOLEAN;
DECLARE rname VARCHAR(50);
DECLARE fullsql VARCHAR(1000);
DECLARE ratio_cur CURSOR FOR SELECT ratio_name FROM ratios;
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_rows = TRUE;
SET fullsql = userquery;
OPEN ratio_cur;
FETCH ratio_cur INTO rname;
rnamewhile: WHILE rname is not null DO
IF INSTR(fullsql, rname) > 0 THEN
SET fullsql = REPLACE(fullsql, rname, CONCAT(' (SELECT `value` FROM companies_ratios INNER JOIN ratios ON companies_ratios.ratio_id = ratios.id WHERE ratios.ratio_name = ''', rname , ''' AND companies_ratios.company_id = cr.company_id ) '));
END IF;
FETCH ratio_cur INTO rname;
IF no_more_rows THEN
CLOSE ratio_cur;
LEAVE rnamewhile;
END IF;
END WHILE rnamewhile;
SET #finalsql = CONCAT('SELECT company_id FROM companies_ratios cr WHERE ', fullsql , ' GROUP BY cr.company_id;');
PREPARE stmt1 FROM #finalsql;
EXECUTE stmt1;
DEALLOCATE PREPARE stmt1;
END$$
I used the following websites to assist creating the query. They should be able to explain the functions a bit more, but basically it loops through the ratios table, and when it finds a ratio name in the user query it replaces it with a select statement. The resulting statement is the joined together at the end to create your final select statement which is then executed, providing a list of unique company id's.
Execute an STRING as a query - Dynamic SQL -> PREPARE
http://forums.mysql.com/read.php?60,27979,30437
MySQL Forums :: Stored Procedures :: MySQL Stored procedure Tutorial
http://forums.mysql.com/read.php?98,358569,358569
MySQL Cursors
http://dev.mysql.com/doc/refman/5.0/en/cursors.html
Loop in Stored Procedures
http://www.mysqltutorial.org/stored-procedures-loop.aspx
SQL Cursor in Stored Procedures
http://www.mysqltutorial.org/sql-cursor-in-stored-procedures.aspx
SQL Syntax for Prepared Statements
http://dev.mysql.com/doc/refman/5.1/en/sql-syntax-prepared-statements.html
CREATE PROCEDURE and CREATE FUNCTION Syntax
http://dev.mysql.com/doc/refman/5.0/en/create-procedure.html

Related

How to count values from the database table column?

I want to convert game tag data found here into a dimension table of a star schema.
But the problem is that steamspy_tag_data table is organised as such every column name is tag name of a game and one game can have multiple tags. For example, lets say I have game Warcraft3 with appid 30 it would be in a table like this.
appid|strategy|action|shooter|fantasy|
-----+--------+------+-------+-------
30 6345 1452 0 6340
Column value greater than 0 signifies amount of user votes that voted certain game to be of that game tag. For Warcraft3 game with appid 30 - 6345 users voted it classifies as strategy 1452 users it classifies as action, .. etc.
Some columns for example "abstract" (column) tag has almost all 0 throughout the whole column meaning almost no game uses that tag, so to simplify 372 columns with over 29k row value into something more compact I want to run a query that would count non-zero value per every tag column and put them in the new table "tagovi" for better visibility which columns(tags) have relatively low game usage count.
so far I came up with this:
CREATE DEFINER=`root`#`localhost` PROCEDURE `zbroji_tagove`()
BEGIN
DECLARE i INTEGER;
DECLARE total_row_count INTEGER;
DECLARE dummy VARCHAR(255);
DECLARE zbrojeno INTEGER;
DECLARE trenutna VARCHAR(255);
DECLARE kursor CURSOR FOR
SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'sppi' AND table_name = 'steamspy_tag_data'
ORDER BY ordinal_position;
SELECT FOUND_ROWS() into total_row_count;
open kursor;
FETCH kursor into dummy;
SET i = 1;
ponavljanje: LOOP
IF i > total_row_count THEN
CLOSE kursor;
LEAVE ponavljanje;
END IF;
FETCH kursor INTO trenutna;
SET zbrojeno = 0;
SET zbrojeno = (SELECT COUNT(*) FROM steamspy_tag_data where trenutna <> 0);
INSERT INTO tagovi(kategorija,broj_igra)
VALUES (
(trenutna),(zbrojeno)
);
SET i = i + 1;
end LOOP;
END
New table tagovi has 3 columns (ID auto_increment, kategorija Varchar(255), broj_igra INTEGER).
When I execute my stored procedure "zbroji_tagove"() I get SQL ERROR CODE 1292; Truncated incorrect DOUBLE value 'some_tag_name'. So somehow sql treats variable value as value instead of column at line SET zbrojeno = (SELECT COUNT() FROM steamspy_tag_data where trenutna <> 0*);
Is there a way for me to accomplish what I want inside MySQL environment?
You can't use variables like that, you need a prepared statement
See
#sql = CONCAT("SELECT COUNT(*) INTO zbrojeno FROM steamspy_tag_data where ",trenutna," <> 0);";
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
DEALLOCATE PREPARE stmt1;

MySQL create view across all prefixed databases' table

I have databases named company_abc, company_xyz, etc. Those company_* databases have all the same structure and they contain users table.
What I need to do is to aggregate all users data from just company_* databases and replicate this view to another server. The view would just be something like
COMPANY NAME | USERNAME
abc | user#email.com
abc | user1#email.com
xyz | user2#email.com
company3 | user3#email.com
Is something like that possible in MySQL?
The databases are created dynamically, as well as the users so I can't create a view with just a static set of databases.
As you say you want to create view with dynamic database names - so the result you want to achieve is not possible in current versions of mysql.
So you have example following options:
Option 1
If you want to get result of all databases users tables you could define a stored procedure that uses prepared statement. This procedure needs parameter db_prefix what in your case is company_%. Basicly this procedure selects all tables named as users from information_schema when database name is like db_prefix parameter value. After that it loops through results and creates query string as union all users tables and executes this query. When creating a query string i also add field called source, so i can identify from what database this result is coming. In my example my databases are all in default collation utf8_unicode_ci.
In this case you can define procedure example "getAllUsers"
-- Dumping structure for procedure company_abc1.getAllUsers
DELIMITER //
CREATE DEFINER=`root`#`localhost` PROCEDURE `getAllUsers`(IN `db_prefix` TEXT)
DETERMINISTIC
COMMENT 'test'
BEGIN
DECLARE qStr TEXT DEFAULT '';
DECLARE cursor_VAL VARCHAR(255) DEFAULT '';
DECLARE done INTEGER DEFAULT 0;
DECLARE cursor_i CURSOR FOR SELECT DISTINCT (table_schema) FROM information_schema.tables WHERE table_name = 'users' AND table_schema LIKE db_prefix COLLATE utf8_unicode_ci;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cursor_i;
read_loop: LOOP
FETCH cursor_i INTO cursor_VAL;
IF done = 1 THEN
LEAVE read_loop;
END IF;
IF qStr != '' THEN
SET qStr = CONCAT(qStr, ' UNION ALL ');
END IF;
SET qStr = CONCAT(qStr, ' SELECT *, \'', cursor_VAL ,'\' as source FROM ', cursor_VAL, '.users');
END LOOP;
CLOSE cursor_i;
SET #qStr = qStr;
PREPARE stmt FROM #qStr;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET #qStr = NULL;
END//
DELIMITER ;
Now you can get your all users result as:
CALL getAllUsers('company_%');
In my example database it results as:
id name source
1 User 1 company_abc1
2 User 2 company_abc1
3 User 3 company_abc1
1 User 1 company_abc2
2 User 2 company_abc2
3 User 3 company_abc2
1 User 1 company_abc3
2 User 2 company_abc3
3 User 3 company_abc3
1 User 1 company_abc4
2 User 2 company_abc4
3 User 3 company_abc4
1 User 1 company_abc5
2 User 2 company_abc5
3 User 3 company_abc5
Option 2
If you really, really need view then you can modify first procedure and instead of executeing select you can create view. Example like this:
-- Dumping structure for procedure company_abc1.createAllUsersView
DELIMITER //
CREATE DEFINER=`root`#`localhost` PROCEDURE `createAllUsersView`(IN `db_prefix` TEXT)
DETERMINISTIC
COMMENT 'test'
BEGIN
DECLARE qStr TEXT DEFAULT '';
DECLARE cursor_VAL VARCHAR(255) DEFAULT '';
DECLARE done INTEGER DEFAULT 0;
DECLARE cursor_i CURSOR FOR SELECT DISTINCT (table_schema) FROM information_schema.tables WHERE table_name = 'users' AND table_schema LIKE db_prefix COLLATE utf8_unicode_ci;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cursor_i;
read_loop: LOOP
FETCH cursor_i INTO cursor_VAL;
IF done = 1 THEN
LEAVE read_loop;
END IF;
IF qStr != '' THEN
SET qStr = CONCAT(qStr, ' UNION ALL ');
END IF;
SET qStr = CONCAT(qStr, ' SELECT *, \'', cursor_VAL ,'\' as source FROM ', cursor_VAL, '.users');
END LOOP;
CLOSE cursor_i;
SET #qStr = CONCAT('CREATE OR REPLACE VIEW allUsersView AS ', qStr);
PREPARE stmt FROM #qStr;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET #qStr = NULL;
END//
DELIMITER ;
In this stored procedure we create/replace view called allUsersView, so basicly every time you will execute this procedure it will updates view.
In my test case it creates view like this:
CREATE OR REPLACE VIEW `allusersview` AS
SELECT *, 'company_abc1' as source FROM company_abc1.users
UNION ALL SELECT *, 'company_abc2' as source FROM company_abc2.users
UNION ALL SELECT *, 'company_abc3' as source FROM company_abc3.users
UNION ALL SELECT *, 'company_abc4' as source FROM company_abc4.users
UNION ALL SELECT *, 'company_abc5' as source FROM company_abc5.users ;
And now you can use view.
SELECT * FROM allusersview
And result is same as in first option.
All tested on:
Mysql 5.6.16
To find the list of database names:
SELECT SCHEMA_NAME
FROM information_schema.`SCHEMATA`
WHERE SCHEMA_NAME LIKE 'company%';
If you can code in something like PHP, the rest is pretty easy -- build a UNION of SELECTs from each database. But, if you must do it just in SQL...
To build the UNION, write a Stored Procedure. It will do the above query in a CURSOR. Inside the loop that walks through the cursor, CONCAT() a constructed SELECT onto a UNION you are building.
When the loop is finished, PREPARE and EXECUTE the constructed UNION. That will deliver something like the output example you had.
But, if you now need to INSERT the results of that into another server, you should leave the confines of the Stored Procedure and use some other language.
OK, OK, if you must stay in SQL, then you need some setup: Create a "Federated" table that connects to the other server. Now, in your SP, concatenate INSERT INTO fed_tbl in front of the UNION. Then the execute should do the entire task.
If you have trouble with the FEDERATED Engine, you may need to switch to FederatedX in MariaDB.
"The details are left as an exercise to the reader."
I already marked this as duplicate of Mysql union from multiple database tables
(SELECT *, 'abc' as COMPANY_NAME from company_abc.users)
union
(SELECT *, 'xyz' as COMPANY_NAME from company_xyz.users)
union
(SELECT *, 'company3' as COMPANY_NAME from company_company3.users)
...
I think that the only method to make this is to write a stored procedure that read all database and table name from information_schema.table, build a string with union select * from company_abc.users union all select * from company_xyz and then execute the command with prepared statement: http://dev.mysql.com/doc/refman/5.0/en/sql-syntax-prepared-statements.html

MySql: Can I Create array using sql

I am Using Mysql DB. my question is : can i create array using sql?
if Yes then how and how to populate this array with output of following query -- "Select column_name1 From tableName".
Help me, Thanks in Advance
As I mentioned in my comment, MySQL does not support arrays by itself. That kind of structures are supported by other programming languages (like PHP, Java, Python, etcetera) and you can write a program capable of connecting to a MySQL database, read data from it and populate arrays (I think PostgreSQL supports an array data type, but I'm not sure).
What you can do is use cursors in a stored procedure to retreive data from a query and store it into variables.
Example:
delimiter $$
create procedure my_procedure()
begin
declare value varchar(100);
declare done int default false;
declare cur cursor for
select column_name1 from your_table;
declare continue handler for not found set done = true;
open cur; -- This line will open the row set and place the cursor
-- on the first row.
loop_data: loop
fetch cur into value; -- This line will fetch the current row
-- into the variable and move the cursor
-- to the next row.
if done then -- If there are no more rows in the
leave loop_data; -- row set, the loop is terminated here
end if; -- and the execution moves to the next
-- instruction after "end loop;"
-- do whatever you need to do with the retrieved value
end loop;
close cur;
end $$
delimiter ;
If you want to use an array in a high level programming language, you can do it using the appropriate methods. Here's an example using Java (read The Java tutorials: JDBC Database access for more info):
public class SomeClass {
/*
Retrieve data from a database and return an array with it.
Parameters:
- conn: Connection to the database.
*/
public String[] getValues(Connection conn) {
String[] ans = new String[10];
int i;
try(
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery(
"select column_name1 from your_table limit 10"
);
) {
rs.beforeFirst();
i = 0;
while(rs.next()) {
ans[i] = rs.getString("column_name1");
i++;
}
} catch(SQLException e) {
// Code to handle the SQL exception
}
return ans;
}
}
References:
MySQL reference manual: Cursors
You can use an variable to simply select your row values into a string. Not precisely an array, but it allows you to store all your values into a single variable:
-- load test data
create table tableName (column_name1 varchar(5));
insert into tableName values
('abcde');
insert into tableName values
('fghij');
insert into tableName values
('klmno');
insert into tableName values
('pqrst');
insert into tableName values
('uvwzy');
insert into tableName values
('z');
-- build "array"
set #array := '';
select #array := concat(#array,column_name1) as array
from tableName;
select #array;

MySql quick user select

I have a Users table and I'm getting user details (usual way) with id.
This is how I'm getting user details;
select id,kullaniciadi,FotoGet(K.Foto,K.Gender) from kullanicilar K where id=1;
FotoGet function always returning string value like 'Photos/ssss.jpg'
So far I have not trouble to use this way but I'm wondering how can I do this with a single function?
Like
set #uid=1;
Select UserGet(#uid);
I will put the id inside parentheses and the function will run my select query and return all user detail columns. id, kullaniciadi, FotoGet(K.id,K.Foto,K.Gender)
Can I use this way to get details?
A MySQL stored function cannot return multiple columns, it can only return a single scalar value.
But you could design a stored procedure that returns a result set:
CREATE PROCEDURE UserGet(uid INT)
SELECT id, kullaniciadi, FotoGet(K.Foto,K.Gender) FROM kullanicilar K WHERE id=uid;
Then call it like so:
CALL UserGet(1234);
It returns a result set just like a SELECT query would.
DELIMITER //
CREATE PROCEDURE UserGet(IN uid INT, IN column_name VARCHAR(64))
BEGIN
SET #sql = CONCAT('SELECT id, ', column_name, ' FROM kullanicilar K WHERE id=?');
PREPARE stmt FROM #sql;
SET #uid = uid;
EXECUTE stmt USING #uid;
END//
DELIMITER ;
Call it like so:
CALL UserGet(1234, 'kullaniciadi');
Remember that it's your responsibility to write code to pass a valid column name as the procedure argument. If you allow untrustworthy content to be passed, then it might be an SQL injection vulnerability.
Re your additional comment:
This should work fine:
CALL UserGet(1234, 'FotoGet(Foto,Gender)');

Using result of SQL Query as table name in mysql trigger

I've to write a trigger on my table which will perform the following functions.
Before Update on row, check price of Item
If price has changed from the last price, then select the table name, where to insert the item name, from another table having type of item and the associated table name.
Insert the item name in the selected table.
To put simply i've a table(TypeNameTable) having item categories and corresponding table names, if the price of item has changed then i've to get the table name from the TypeNameTable and insert the item name in the table, which is retrieved from TypeNameTable.
I'm not able to insert into table when I get the table names dynamically. Please suggest how to do it. Here's what I'm doing:
BEGIN
#declare countryTableName varchar(50);
declare itemPrice int;
declare itemTableName text;
IF (New.Price != Old.Price) THEN
SET countryTableName = (select `ItemManager`.`TypeNames`.`TypeTableName`
from `ItemManager`.`TypeNames`
where `ItemManager`.`TypeNames`.`ItemType` = NEW.ItemType);
INSERT INTO `ItemManager`.itemTableName
( `ItemName`, `ItemPrice`,
VALUES
( NEW.Name, New.Price );
END IF;
END$$
I get the error
ItemManager.itemTableName doesn't exists.
Answering my own question.
Figured out that using Dynamic SQL is not allowed in MySQL triggers . The restrictions are listed here.
However it's possible in Oracle where we can use PRAGMA AUTONOMOUS_TRANSACTION which executes the query in new context, and hence supports Dynamic SQL.
Example listed here at Point 27 .
You could CONCAT() your INSERT statement into a variable and execute that as PREPARED STATEMENT, someting like
...
SET #sql := CONCAT( 'INSERT INTO ', itemTableName, ' ... ' );
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
DEALLOCATE PREPARE stmt1;
...
afaik this is the only way to process dynamically generated SQL in stored routines and triggers.
If it is possible, I'd suggest you to change design a little. Instead of different tables you can create one table itemTable.
...
IF (New.Price != Old.Price) THEN
INSERT INTO `ItemManager`.`itemTable`
( `ItemName`, `ItemPrice`,
VALUES
( NEW.Name, New.Price );
END IF;
...
If there are different item properties, this table can be a parent table for specific child tables.