I'll start this off by saying I know that there are more practical ways to solve this. It's more of an intellectual curiosity than anything else.
I've inherited a MySQL database where some columns are stored as varchar(5) but actually contain the literals "True" or "False". Changing the structure of the data is not an option right now due to other issues. I'm mapping the columns to an ORM (SQLAlchemy), and I want the column to be mapped to a Boolean data type in the supporting codebase using a type adapter. (I've written this adapter already; it's not the problem.)
To help make the mapping process faster, I'm writing a small query to look at the INFORMATION_SCHEMA table and build a line of Python code defining the column using the ORM's syntax. I cannot assume that the data type varchar(5) is a Boolean column - I need to inspect the contents of that column to see if there are values contained in it besides True and False.
Can I write a query that will both get the column type from INFORMATION_SCHEMA and check the actual values stored in that column?
Here is the query I have so far:
SELECT CONCAT(
"Column(""",
col.column_name,
""", ",
(CASE
WHEN col.DATA_TYPE = "int" THEN "Integer"
-- Code in question
WHEN
col.DATA_TYPE = "varchar"
AND col.CHARACTER_MAXIMUM_LENGTH = 5
AND NOT EXISTS(
-- Doesn't seem to work
SELECT DISTINCT col.COLUMN_NAME
FROM col.TABLE_NAME
WHERE col.COLUMN_NAME NOT IN ("True", "False")
)
THEN "BoolStoredAsVarchar"
WHEN col.DATA_TYPE = "varchar" THEN CONCAT("String(", col.CHARACTER_MAXIMUM_LENGTH, ")")
-- Default if it's not a recognized column type
ELSE col.DATA_TYPE
END),
"),"
) AS alchemy
FROM information_schema.columns AS col
WHERE
col.TABLE_SCHEMA = "my_schema"
AND col.TABLE_NAME = "my_table"
ORDER BY col.ORDINAL_POSITION;
Running this code gives me a permissions error: Error Code: 1142. SELECT command denied to user 'user'#'host' for table 'table_name'. Presumably it's trying to use col.TABLE_NAME as a literal instead of interpreting it.
I've also tried creating a simple stored procedure and making table_name into a variable. However, replacing the FROM clause inside the EXISTS with a variable name gives me a syntax error instead.
Again, it's easy enough to run the query myself to see what's in that column. I'd just like to know if this is possible, and if so, how to do it.
You can't do what you're trying to do in a single query.
The reason is that table names (or any other identifier) must be fixed in the query at the time it is parsed, which is before it has read any values from tables. Thus you can't read the name of a table as a string from information_schema and also read from the table with that name in the same query.
You must read the table name from information_schema and then use that result to format a second query.
This isn't a problem specific to MySQL. It's true of any SQL implementation.
Is it possible to achieve something like this?
Suppose name and plural_name are fields of Animal's table.
Suppose pluralise_animal is a helper function which takes a string and returns its plural literal.
I cannot loop over the animal records for technical reasons.
This is just an example
Animal.update_all("plural_name = ?", pluralise_animal("I WANT THE ANIMAL NAME HERE, the `name` column's value"))
I want something similar to how you can use functions in MySQL while modifying column values. Is this out-of-scope or possible?
UPDATE animals SET plural_name = CONCAT(name, 's') -- just an example to explain what I mean by referencing a column. I'm aware of the problems in this example.
Thanks in advance
I cannot loop over the animal records for technical reasons.
Sorry, this cannot be done with this restriction.
If your pluralizing helper function is implemented in the client, then you have to fetch data values back to the client, pluralize them, and then post them back to the database.
If you want the UPDATE to run against a set of rows without fetching data values back to the client, then you must implement the pluralization logic in an SQL expression, or a stored function or something.
UPDATE statements run in the database engine. They cannot call functions in the client.
Use a ruby script to generate a SQL script that INSERTS the plural values into a temp table
File.open(filename, 'w') do |file|
file.puts "CREATE TEMPORARY TABLE pluralised_animals(id INT, plural varchar(50));"
file.puts "INSERT INTO pluralised_animals(id, plural) VALUES"
Animal.each.do |animal|
file.puts( "( #{animal.id}, #{pluralise_animal(animal.name)}),"
end
end
Note: replace the trailing comma(,) with a semicolon (;)
Then run the generated SQL script in the database to populate the temp table.
Finally run a SQL update statement in the database that joins the temp table to the main table...
UPDATE animals a
INNER JOIN pluralised_animals pa
ON a.id = pa.id
SET a.plural_name = pa.plural;
I am needing to use concat to create custom sql statements.
I'm trying to use
update setuptable set image_loc = CONCAT('/file/to/image/', (select UID from table1), 'nam.jpg')
The goal is to update the image_loc in setuptable to "/file/to/image/UIDnam.jpg"
I'm getting Syntax errors and "Expected End of Statement".
How can i tie a simple string, a subquery, and another simple string together to update a field?
I found a way to make it work. If any one has any tips of improvement, I'm up to learn!
*I can't use variables
update setuptable set image_loc = CONCAT('/file/to/image/', (select UID from table1)+, 'nam.jpg')
I found slightly better following structure :
update setuptable
set image_loc = CONCAT('/file/to/image/', t1.UID, 'nam.jpg')
from table1 t1
I don't have mysql server at hand for the moment, so I have not tested.
In BIRT, When i try to fetch the records from my localhost, its working fine. But when i try to work with remote connection i am getting error as specified below:
Error :
org.eclipse.birt.data.engine.odaconsumer.OdaDataException: Cannot get the result set metadata.
org.eclipse.birt.report.data.oda.jdbc.JDBCException: SQL statement does not return a ResultSet object.
SQL error #1:Table 'test.TBLUSERS' doesn't exist ... 63 more
Caused by: com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: Table 'testbms.TBLUSERS' doesn't exist
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2985)
Note:
Tablenames are automatically changing to capital letters, is that because of it.
Because client server is linux and is it acting with case sensitive.
But it displays column names but not the records. As soon as i click
on finish, i get the error as specified in the below images.
Reference Image:
As you can see in the above image, it has populated the table columns in the second row
Is their any special configurations need to be done for remote connection or am i doing anything wrong?
As you stated, it is probably a case of case-sensitivity:
http://dev.mysql.com/doc/refman/5.0/en/identifier-case-sensitivity.html
Although database and table names are not case sensitive on some
platforms, you should not refer to a given database or table using
different cases within the same statement. The following statement
would not work because it refers to a table both as my_table and as
MY_TABLE: mysql> SELECT * FROM my_table WHERE MY_TABLE.col=1;
If your development box isn't case sensitive then when you change the case of your tablename to match that on production you'll still be able to test. There might also be a way in MySQL using system tables. (See the following query for an example of querying to see if a table exists.):
SELECT count(*)
FROM information_schema.tables
WHERE table_schema = <schema-or-db-name>
AND table_name = <table-or-view-name>
but more realistically, your target database should be passed to your report through a variable that you can check in the scripting of the dataset. Set the "this.query" value to equal the appropriate query based on that variable's value.
E.G.:
if ( params["source_db"].value == "Server=myProductionAddress;Database=myProductionDB;Uid=myUsername;Pwd=myPassword;" )
{
this.query = "SELECT .... prodTableName";
}
else
{
this.query = "SELECT .... devTableName";
}
I just asked an SQL related question, and the first answer was: "This is a situation where dynamic SQL is the way to go."
As I had never heard of dynamic SQL before, I immediately searched this site and the web for what it was. Wikipedia has no article with this title. The first Google results all point to user forums where people ask more or less related questions.
However, I didn't find a clear definition of what a 'dynamic SQL' is. Is it something vendor specific? I work with MySQL and I didn't find a reference in the MySQL handbook (only questions, mostly unanswered, in the MySQL user forums).
On the other hand, I found many references to stored procedures. I have a slightly better grasp of what stored procedures are, although I have never used any. How are the two concepts related? Are they the same thing or does one uses the other?
Basically, what is needed is a simple introduction to dynamic SQL for someone who is new to the concept.
P.S.: If you feel like it, you may have a go at answering my previous question that prompted this one: SQL: How can we make a table1 JOIN table2 ON a table given in a field in table1?
Dynamic SQL is merely where the query has been built on the fly - with some vendors, you can build up the text of the dynamic query within one stored procedure, and then execute the generated SQL. In other cases, the term merely refers to a decision made by code on the client (this is at least vendor neutral)
Other answers have defined what dynamic SQL is, but I didn't see any other answers that attempted to describe why we sometimes need to use it. (My experience is SQL Server, but I think other products are generally similar in this respect.)
Dynamic SQL is useful when you are replacing parts of a query that can't be replaced using other methods.
For example, every time you call a query like:
SELECT OrderID, OrderDate, TotalPrice FROM Orders WHERE CustomerID = ??
you will be passing in a different value for CustomerID. This is the simplest case, and one that can by solved using a parameterized query, or a stored procedure that accepts a parameter, etc.
Generally speaking, dynamic SQL should be avoided in favor of parameterized queries, for performance and security reasons. (Although the performance difference probably varies quite a bit between vendors, and perhaps even between product versions, or even server configuration).
Other queries are possible to do using parameters, but might be simpler as dynamic SQL:
SELECT OrderID, OrderDate, TotalPrice FROM Orders
WHERE CustomerID IN (??,??,??)
If you always had 3 values, this is as easy as the first one. But what if this is a variable-length list? Its possible to do with parameters, but can be very difficult. How about:
SELECT OrderID, OrderDate, TotalPrice FROM Orders WHERE CustomerID = ??
ORDER BY ??
This can't be substituted directly, you can do it with a huge complicated CASE statement in the ORDER BY explicitly listing all possible fields, which may or may not be practical, depending on the number of fields available to sort by.
Finally, some queries simply CAN'T be done using any other method.
Let's say you have a bunch of Orders tables (not saying this is great design), but you might find yourself hoping you can do something like:
SELECT OrderID, OrderDate, TotalPrice FROM ?? WHERE CustomerID = ??
This can't be done using any other methods. In my environment, I frequently encounter queries like:
SELECT (programatically built list of fields)
FROM table1 INNER JOIN table2
(Optional INNER JOIN to table3)
WHERE (condition1)
AND (long list of other optional WHERE clauses)
Again, not saying that this is necessarily great design, but dynamic SQL is pretty much required for these types of queries.
Hope this helps.
Dynamic SQL is simply a SQL statement that is composed on the fly before being executed. For example, the following C# (using a parameterized query):
var command = new SqlCommand("select * from myTable where id = #someId");
command.Parameters.Add(new SqlParameter("#someId", idValue));
Could be re-written using dynamic sql as:
var command = new SqlCommand("select * from myTable where id = " + idValue);
Keep in mind, though, that Dynamic SQL is dangerous since it readily allows for SQL Injection attacks.
Dynamic SQL is a SQL built from strings at runtime. It is useful to dynamically set filters or other stuff.
An example:
declare #sql_clause varchar(1000)
declare #sql varchar(5000)
set #sql_clause = ' and '
set #sql = ' insert into #tmp
select
*
from Table
where propA = 1 '
if #param1 <> ''
begin
set #sql = #sql + #sql_clause + ' prop1 in (' + #param1 + ')'
end
if #param2 <> ''
begin
set #sql = #sql + #sql_clause + ' prop2 in (' + #param2 + ')'
end
exec(#sql)
It is exactly what Rowland mentioned. To elaborate on that a bit, take the following SQL:
Select * from table1 where id = 1
I am not sure which language you are using to connect to the database, but if I were to use C#, an example of a dynamic SQL query would be something like this:
string sqlCmd = "Select * from table1 where id = " + userid;
You want to avoid using dynamic SQL, because it becomes a bit cumbersome to keep integrity of the code if the query get too big. Also, very important, dynamic SQL is susceptible to SQL injection attacks.
A better way of writing the above statement would be to use parameters, if you are using SQL Server.
Rowland is correct, and as an addendum, unless you're properly using parameters (versus just concatonating parameter values inline from provided text, etc.) it can also be a security risk. It's also a bear to debug, etc.
Lastly, whenever you use dynamic SQL unwisely, things are unleashed and children are eaten.
To most databases, every SQL query is "dynamic" meaning that it is a program that is interpreted by the query optimiser given the input SQL string and possibly the parameter bindings ("bind variables").
Static SQL
However, most of the time, that SQL string is not constructed dynamically but statically, either in procedural languages like PL/SQL:
FOR rec IN (SELECT * FROM foo WHERE x = 1) LOOP
-- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ "static SQL"
..
END LOOP;
Or in client / host languages like Java, using JDBC:
try (ResultSet rs = stmt.executeQuery("SELECT * FROM foo WHERE x = 1")) {
// "static SQL" ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
..
}
In both cases, the SQL string is "static" in the language that embeds it. Technically, it will still be "dynamic" to the SQL engine, which doesn't know how the SQL string is constructed, nor that it was a static SQL string.
Dynamic SQL
Sometimes, the SQL string needs to be constructed dynamically, given some input parameters. E.g. the above query might not need any predicate at all in some cases.
You might then choose to proceed to constructing the string dynamically, e.g. in PL/SQL:
DECLARE
TYPE foo_c IS REF CURSOR;
v_foo_c foo_c;
v_foo foo%ROWTYPE;
sql VARCHAR2(1000);
BEGIN
sql := 'SELECT * FROM foo';
IF something THEN
sql := sql || ' WHERE x = 1'; -- Beware of syntax errors and SQL injection!
END IF;
OPEN v_foo_c FOR sql;
LOOP
FETCH v_foo_c INTO v_foo;
EXIT WHEN v_foo_c%NOTFOUND;
END LOOP;
END;
Or in Java / JDBC:
String sql = "SELECT * FROM foo";
if (something)
sql += " WHERE x = 1"; // Beware of syntax errors and SQL injection!
try (ResultSet rs = stmt.executeQuery(sql)) {
..
}
Or in Java using a SQL builder like jOOQ
// No syntax error / SQL injection risk here
Condition condition = something ? FOO.X.eq(1) : DSL.trueCondition();
for (FooRecord foo : DSL.using(configuration)
.selectFrom(FOO)
.where(condition)) {
..
}
Many languages have query builder libraries like the above, which shine most when doing dynamic SQL.
(Disclaimer: I work for the company behind jOOQ)
Is it something vendor specific?
The SQL-92 Standard has a whole chapter on dynamic SQL (chapter 17) but it only applies to FULL SQL-92 and I know of no vendor that has implemented it.
I think what's meant is that you should build the query dynamically before executing it. For your other questions this means that you should select the table name you need first and the use your programming language to build a second query for doing what you want (what you want to do in the other question isn't possible directly like you want).