This is my timestamp
q1<-Sys.time()-777000
q1
#"2019-09-12 08:39:27 GMT"
This is what I am trying to do , am getting an error
Sys.time()
dbSendQuery(conn,"delete from anomaly_hourly_temp where report_time>q1")
Sys.time()
dbSendQuery(conn,"delete from anomaly_hourly_temp where report_time>q1")
Error in .local(conn, statement, ...) :
could not run statement: Unknown column 'q1' in 'where clause'
Also tried this , though it does not show any error , but it does not delete any row based on timestamps
Sys.time()
dbSendQuery(conn,"delete from anomaly_hourly_temp where report_time>'q1'")
Sys.time()
If I explicitly specify timestamp(q1) , it does work as given below
dbSendQuery(conn,"delete from anomaly_hourly_temp where report_time>'2019-09-22 11:42:51'")
Right now you attempt to interpolate the R variable into the SQL statement but SQL reads the literal q1 and not its underlying value. While concatenating the R variable into SQL string is a solution, it is safer, more efficient, avoids quotes, and industry best practice to run parameterization using a prepared statement with parameter binded in subsequent step:
# PREPARED STATEMENT
sql <- "delete from anomaly_hourly_temp where report_time > ?")
# BIND PARAM AND EXECUTE ACTION
dbSendQuery(conn, sql, list(q))
Use paste0 to paste the query together.
DBI::dbSendQuery(conn,
paste0("delete from anomaly_hourly_temp where report_time > '", q1, "'"))
Similar to paste0, we can also use paste/str_c/glue/sprintf or other function which helps to paste the query together.
Related
I'll start this off by saying I know that there are more practical ways to solve this. It's more of an intellectual curiosity than anything else.
I've inherited a MySQL database where some columns are stored as varchar(5) but actually contain the literals "True" or "False". Changing the structure of the data is not an option right now due to other issues. I'm mapping the columns to an ORM (SQLAlchemy), and I want the column to be mapped to a Boolean data type in the supporting codebase using a type adapter. (I've written this adapter already; it's not the problem.)
To help make the mapping process faster, I'm writing a small query to look at the INFORMATION_SCHEMA table and build a line of Python code defining the column using the ORM's syntax. I cannot assume that the data type varchar(5) is a Boolean column - I need to inspect the contents of that column to see if there are values contained in it besides True and False.
Can I write a query that will both get the column type from INFORMATION_SCHEMA and check the actual values stored in that column?
Here is the query I have so far:
SELECT CONCAT(
"Column(""",
col.column_name,
""", ",
(CASE
WHEN col.DATA_TYPE = "int" THEN "Integer"
-- Code in question
WHEN
col.DATA_TYPE = "varchar"
AND col.CHARACTER_MAXIMUM_LENGTH = 5
AND NOT EXISTS(
-- Doesn't seem to work
SELECT DISTINCT col.COLUMN_NAME
FROM col.TABLE_NAME
WHERE col.COLUMN_NAME NOT IN ("True", "False")
)
THEN "BoolStoredAsVarchar"
WHEN col.DATA_TYPE = "varchar" THEN CONCAT("String(", col.CHARACTER_MAXIMUM_LENGTH, ")")
-- Default if it's not a recognized column type
ELSE col.DATA_TYPE
END),
"),"
) AS alchemy
FROM information_schema.columns AS col
WHERE
col.TABLE_SCHEMA = "my_schema"
AND col.TABLE_NAME = "my_table"
ORDER BY col.ORDINAL_POSITION;
Running this code gives me a permissions error: Error Code: 1142. SELECT command denied to user 'user'#'host' for table 'table_name'. Presumably it's trying to use col.TABLE_NAME as a literal instead of interpreting it.
I've also tried creating a simple stored procedure and making table_name into a variable. However, replacing the FROM clause inside the EXISTS with a variable name gives me a syntax error instead.
Again, it's easy enough to run the query myself to see what's in that column. I'd just like to know if this is possible, and if so, how to do it.
You can't do what you're trying to do in a single query.
The reason is that table names (or any other identifier) must be fixed in the query at the time it is parsed, which is before it has read any values from tables. Thus you can't read the name of a table as a string from information_schema and also read from the table with that name in the same query.
You must read the table name from information_schema and then use that result to format a second query.
This isn't a problem specific to MySQL. It's true of any SQL implementation.
I've written a common table expression to return hierarchical information and it seems to work without issue if I hard code a value into the WHERE statement. If I use a variable (even if the variable contains the same information as the hard coded value), I get the error The maximum recursion 100 has been exhausted before statement completion.
This is easier shown with a simple example (note, I haven't included the actual code for the CTE just to keep things clearer. If you think it's useful, I can certainly add it).
This Works
WITH Blder
AS
(-- CODE IS HERE )
SELECT
*
FROM Blder as b
WHERE b.PartNo = 'ABCDE';
This throws the Max Recursion Error
DECLARE #part CHAR(25);
SET #part = 'ABCDE'
WITH Blder
AS
(-- CODE IS HERE )
SELECT
*
FROM Blder as b
WHERE b.PartNo = #part;
Am I missing something silly? Or does the SQL engine handle hardcoded values and parameter values differently in this type of scenario?
Kindly put semicolon at the end of your variable assignment statement
SET #part ='ABCDE';
Your SELECT statement is written incorrectly: the SQL Server Query Optimizer is able to optimize away the potential cycle if fed the literal string, but not when it's fed a variable, which uses the plan that developed from the statistics.
SQL Server 2016 improved on the Query Optimizer, so if you could migrate your DB to SQL Server 2016 or newer, either with the DB compatibility level set to 130 or higher (for SQL Server 2016 and up), or have it kept at 100 (for SQL Server 2008) but with OPTION (USE HINT ('ENABLE_QUERY_OPTIMIZER_HOTFIXES')) added to the bottom of your SELECT statement, you should get the desired result without the max recursion error.
If you are stuck on SQL Server 2008, you could also add OPTION (RECOMPILE) to the bottom of your SELECT statement to create an ad hoc query plan that would be similar to the one that worked correctly.
I'm currently using the mySQL common schema package along with the split() function, but I'm struggling to get a working JOIN query to work?
set #script := "
split({size:2000} :
UPDATE world
SET world.CountryName = (
SELECT country.nicename
FROM country
WHERE country.iso = world.Country
)
)
{
throttle 4;
SELECT $split_total_rowcount AS 'rows updated so far';
}
";
call common_schema.run(#script);
When running this query, it produces the following:
#1644 - QueryScript error: [split() cannot deduce split table name. Please specify explicitly] at 34: "UPDATE world
SET world.Country
As for why I'm trying to split my UPDATE query into chunks, is because it's trying to update a table that's got 3M+ rows & is struggling when doing the query on it's own
Please specify explicitly appears to refer to using this format:
Multiple tables operations; explicit declaration of splitting table:
split (schema_name.table_name: statement operating on multiple tables)
statement;
https://shlomi-noach.github.io/common_schema/query_script_split.html
See also explicit declaration.
I have been trying to write an R script to query Impala database. Here is the query to the database:
select columnA, max(columnB) from databaseA.tableA where columnC in (select distinct(columnC) from databaseB.tableB ) group by columnA order by columnA
When I run this query manually (read: outside the Rscript via impala-shell), I am able to get the table contents. However, when the same is tried via the R script, I get the following error:
[1] "HY000 140 [Cloudera][ImpalaODBC] (140) Unsupported query."
[2] "[RODBC] ERROR: Could not SQLExecDirect 'select columnA, max(columnB) from databaseA.tableA where columnC in (select distinct(columnC) from databaseB.tableB ) group by columnA order by columnA'
closing unused RODBC handle 1
Why does the query fail when tried via R? and how do I fix this? Thanks in advance :)
Edit 1:
The connection script looks as below:
library("RODBC");
connection <- odbcConnect("Impala");
query <- "select columnA, max(columnB) from databaseA.tableA where columnC in (select distinct(columnC) from databaseB.tableB ) group by columnA order by columnA";
data <- sqlQuery(connection,query);
You need to install the relevant drivers, please look at the following link
I had the same issue, all i had to do was update the ODBC drivers.
Also if you can update your odbcConnect with the username and password
connection <- odbcConnect("Impala");
to
connection <- odbcConnect("Impala", uid="root", pwd="password")
The RODBC package is quirky: if there's no row updated/deleted in the query execution it will throw an error.
So before using sqlDelete to delete rows, or using sqlUpdate to update values, first check if there's at least one row that will be deleted/updated by querying COUNT(*).
I've had no problem after implementing the check, for Oracle SQL 12g.
An alternative would be to use a staging table for the new batch of data, and use sqlQuery to execute a MERGE command. RODBC won't complaint if there's zero row merged.
This might also be due to an error in your sql query itself. For example, I got this error when I missed an 'in' in the following generalized statement. Example:
stringstuff <- someDT$columnyouwanttouse
somestring <- toString(sprintf("'%s'", stringstuff))
RESULTS <- sqlQuery(con, paste0("select
fling as flam
and toot **in** (",somestring,")
limit 30
;"))
I got the error you did when I left out the 'in', so double check your syntax.
This error message can arise if the table doesn't exist in the database.
A few sensible checks:
Check for typos in the table name in your query
See if you can run the same query on the same database via another sql client
Talk to your data base administrator to confirm that the table does exist
Re-installing the RODBC package did the trick for me!
I had a similar problem. After unnisntalling the R version 4.2.1 and install the R version 4.1.3 the problem was solved.
Any assistance here would be great.
I am trying to use parameters to dynamically change 'ORDER BY'
Below is the code I have tried but despite following the documentation I still get an error '[FIREDAC][PHYS][MYSQL] You have an error in your SQL syntax ... near "ORDER BY some_field" at line 4'
I have set ParamCreate to True
My database is MySQL
FDQuery1.Close;
FDQuery1.SQL.Clear;
FDQuery1.SQL.Add('SELECT *');
FDQuery1.SQL.Add('FROM my_table');
FDQuery1.SQL.Add('LIMIT 1000');
FDQuery1.SQL.Add(':id');
FDQuery1.ParamByName('id').AsString := 'ORDER BY some_field';
FDQuery1.Open;
You did not cite the exception message as it shows up. Here is the original message
[FireDAC][Phys][MySQL] You have an error in your SQL syntax ... near ''ORDER BY some_field'' at line 4.
compare to your cite
[FIREDAC][PHYS][MYSQL] You have an error in your SQL syntax ... near "ORDER BY some_field" at line 4
To avoid this for the future just press CTRL+C on the focused exception window and the complete message is inside your clipboard and can be pasted wherever you like
Now reading this, the error is now very clear.
You expect to get a statement like this
SELECT *
FROM my_table
LIMIT 1000
ORDER BY some_field
But using the parameter you will get the following statement
SELECT *
FROM my_table
LIMIT 1000
'ORDER BY some_field'
and that is exactly what the exception message is telling you.
Just check the exception message with the previous statement
... near 'ORDER BY some_field' at line 4.
and
... near ''ORDER BY some_field'' at line 4.
As a conclusion it is not possible to change the statement itself using parameters. You can only pass values as parameters for the statement.
And the correct statement should be anyway
SELECT *
FROM my_table
ORDER BY some_field
LIMIT 1000
Don't know if this helps.
But you can use the 'Macros' property of TFDQuery, like Parameter that are identified by the ':', the Macros are identified bye the '!', You can also combine Macros and Params. The Macros property works almost as the Params property. Use the TFDQuery.MacroByname to assign a Macro Value, and use the TFDQuery.MacroByname('MacroName').AsRaw to assign a string As-Is.
So your query should look like:
FDQuery1.Close;
FDQuery1.SQL.Text := 'SELECT * FROM !TABLE_NAME !WHERE_CLAUSE !ORDERBY_CLAUSE';
FDQuery.MacroByname('Table_name').AsRaw := 'my_table';
FDQuery.MacroByname('Where_clause').AsRaw := 'WHERE field1 = :ID_Value';
FDQuery.MacroByname('OrderBy_clause').AsRaw := 'ORDER BY field1';
FDQuery.ParamByname('ID_Value').AsInteger := 1;
FDQuery1.Open;
Hope this helps
you need very simple SQL query:
FDQuery1.Close;
FDQuery1.SQL.Text := 'SELECT * FROM my_table';
FDQuery1.Open;
To set a limit of the record count, you can use property of the FDQuery1:
FDQuery1.FetchOptions.RecsMax := 1000;
To sort values you can use
FDQuery1.IndexFieldNames = 'field_name'
or
FDQuery1.IndexFieldNames = 'field_one_name;field_two_name'
instead of your code.
Edit: Oh, I didn't actually answer your question. So for FireDac, you can just set the FDQuery1.IndexFieldNames property to the name of the field/s you want to order by. No need to close and re-open your query, or change the SQL.
Previous answer: You cannot pass SQL code [EDIT: including ORDER BY etc.] in parameters, only parameter values, i.e. integers, strings etc.
This principle is extremely important, otherwise you could pass e.g.
FDQuery1.ParamByName('id').AsString := '; TRUNCATE TABLE my_table';
Executing your query would then delete everything in the table instead of doing what it's supposed to do. Or with a bit more work, your same query could return passwords, credit card numbers or whatever else is in your database. This would have been a huge vulnerability, and is known as SQL Injection. Please see for example:
http://www.w3schools.com/sql/sql_injection.asp
http://sqlmap.org/
http://hackaday.com/2014/09/01/gaining-access-to-the-oculus-developer-database
RXLIB has this functionality. It has the MACRO opttions, where you can write a code like this:
Select %fields_
from %table_
where %condition_
order by %order_
BUT it´s only for use qith BDE.
Will be wonderfull if someone rewrite the code to work with ADO ou FIREDAC.