After trying to determine why my SQL statement was returning an error in the Node code I am refactoring to connect to MariaDB rather than via SQL Anywhere, I have narrowed it down to the REPLACE statement we use to compute how many records to process, and how many to skip.
My initial test SQL SELECT statement looks like this:
SELECT customer_name FROM ar.customers
We then use a REPLACE statement to, as I say, determine how many records to process, and how many to skip. When we were using SQL Anywhere that looked like this:
const sql = this.query.replace(/SELECT/i, `SELECT TOP ${recordsPerRun} START AT ${recordsProcessed + 1}`);
That syntax needs to change because MariaDB uses "LIMIT" instead of "TOP". And from my understanding, the first parameter will be the number of records to skip, and the second one how many to return.
So, in my case, it'd be something like:
LIMIT ${recordsProcessed}, ${recordsPerRun}
However, I can't quite get the full syntax right. How can I write this REPLACE statement in a way that will work with my initial test SQL SELECT statement from above? This seems tricky to do since in MariaDB LIMIT now goes at the end of the query, not at the beginning, like TOP did for MySQL.
LIMIT goes at the end, so there's nothing to replace, just concatenate it:
const sql = this.query + ` LIMIT ${recordsProcessed}, ${recordsPerRun}`;
or combine it into the template:
const sql = `${this.query} LIMIT ${recordsProcessed}, ${recordsPerRun}`;
Related
I've written a common table expression to return hierarchical information and it seems to work without issue if I hard code a value into the WHERE statement. If I use a variable (even if the variable contains the same information as the hard coded value), I get the error The maximum recursion 100 has been exhausted before statement completion.
This is easier shown with a simple example (note, I haven't included the actual code for the CTE just to keep things clearer. If you think it's useful, I can certainly add it).
This Works
WITH Blder
AS
(-- CODE IS HERE )
SELECT
*
FROM Blder as b
WHERE b.PartNo = 'ABCDE';
This throws the Max Recursion Error
DECLARE #part CHAR(25);
SET #part = 'ABCDE'
WITH Blder
AS
(-- CODE IS HERE )
SELECT
*
FROM Blder as b
WHERE b.PartNo = #part;
Am I missing something silly? Or does the SQL engine handle hardcoded values and parameter values differently in this type of scenario?
Kindly put semicolon at the end of your variable assignment statement
SET #part ='ABCDE';
Your SELECT statement is written incorrectly: the SQL Server Query Optimizer is able to optimize away the potential cycle if fed the literal string, but not when it's fed a variable, which uses the plan that developed from the statistics.
SQL Server 2016 improved on the Query Optimizer, so if you could migrate your DB to SQL Server 2016 or newer, either with the DB compatibility level set to 130 or higher (for SQL Server 2016 and up), or have it kept at 100 (for SQL Server 2008) but with OPTION (USE HINT ('ENABLE_QUERY_OPTIMIZER_HOTFIXES')) added to the bottom of your SELECT statement, you should get the desired result without the max recursion error.
If you are stuck on SQL Server 2008, you could also add OPTION (RECOMPILE) to the bottom of your SELECT statement to create an ad hoc query plan that would be similar to the one that worked correctly.
I have the following query
SELECT t.res, IF(t.res=0, "zero", "more than zero")
FROM (
SELECT table.*, IF (RAND()<=0.2,1, IF (RAND()<=0.4,2, IF (RAND()<=0.6,3,0))) AS res
FROM table LIMIT 20) t
which returns something like this:
That's exactly what you would expect. However, as soon as I remove the LIMIT 20 I receive highly unexpected results (there are more rows returned than 20, I cut it off to make it easier to read):
SELECT t.res, IF(t.res=0, "zero", "more than zero")
FROM (
SELECT table.*, IF (RAND()<=0.2,1, IF (RAND()<=0.4,2, IF (RAND()<=0.6,3,0))) AS res
FROM table) t
Side notes:
I'm using MySQL 5.7.18-15-log and this is a highly abstracted example (real query is much more difficult).
I'm trying to understand what is happening. I do not need answers that offer work arounds without any explanations why the original version is not working. Thank you.
Update:
Instead of using LIMIT, GROUP BY id also works in the first case.
Update 2:
As requested by zerkms, I added t.res = 0 and t.res + 1 to the second example
The problem is caused by a change introduced in MySQL 5.7 on how derived tables in (sub)queries are treated.
Basically, in order to optimize performance, some subqueries are executed at different times and / or multiple times leading to unexpected results when your subquery returns non-deterministic results (like in my case with RAND()).
There are two easy (and likewise ugly) workarounds to get MySQL to "materialize" (aka return deterministic results) these subqueries: Use LIMIT <high number> or GROUP BY id both of which force MySQL to materialize the subquery and return the expected results.
The last option is turn off derived_merge in the optimizer_switch variable: derived_merge=off (make sure to leave all the other parameters as they are).
Further readings:
https://mysqlserverteam.com/derived-tables-in-mysql-5-7/
Subquery's rand() column re-evaluated for every repeated selection in MySQL 5.7/8.0 vs MySQL 5.6
This should be the simplest thing but for some reason it's eluding me completely.
I have a Sequel connection to a database named DB. It's using the Mysql2 engine if that's important.
I'm trying to update a single record in a table in the database. The short loop I'm using looks like this:
dataset = DB["SELECT post_id, message FROM xf_post WHERE message LIKE '%#{match}%'"]
dataset.each do |row|
new_message = process_message(row[:message])
# HERE IS WHERE I WANT TO UPDATE THE ROW IN THE DATABASE!
end
I've tried:
dataset.where('post_id = ?', row[:post_id]).update(message: new_message)
Which is what the Sequel cheat sheet recommends.
And:
DB["UPDATE xf_post SET message = ? WHERE post_id = ?", new_message, row[:post_id]]
Which should be raw SQL executed by the Sequel connector. Neither throws an error or outputs any error message (I'm using a logger with the Sequel connection). But both calls fail to update the records in the database. The data is unchanged when I query the database after running the code.
How can I make the update call function properly here?
Your problem is you are using a raw SQL dataset, so the where call isn't going to change the SQL, and update is just going to execute the raw SQL. Here's what you want to do:
dataset = DB[:xf_post].select(:post_id, :message).
where(Sequel.like(:message, "%#{match}%"))
That will make the where/update combination work.
Note that your original code has a trivial SQL injection vulnerability if match depends on user input, which this new code avoids. You may want to consider using Dataset#escape_like if you want to escape metacharacters inside match, otherwise if match depends on user input, it's possible for users to use very complex matching syntax that the database may execute slowly or not handle properly.
Note that the reason that
DB["UPDATE xf_post SET message = ? WHERE post_id = ?", new_message, row[:post_id]]
doesn't work is because it only creates a dataset, it doesn't execute it. You can actually call update on that dataset to run the query and return number of affected rows.
I'm using the Wordnet SQL database from here: http://wnsqlbuilder.sourceforge.net
It's all built fine and users with appropriate privileges have been set.
I'm trying to find synonyms of words and have tried to use the two example statements at the bottom of this page: http://wnsqlbuilder.sourceforge.net/sql-links.html
SELECT synsetid,dest.lemma,SUBSTRING(src.definition FROM 1 FOR 60) FROM wordsXsensesXsynsets AS src INNER JOIN wordsXsensesXsynsets AS dest USING(synsetid) WHERE src.lemma = 'option' AND dest.lemma <> 'option'
SELECT synsetid,lemma,SUBSTRING(definition FROM 1 FOR 60) FROM wordsXsensesXsynsets WHERE synsetid IN ( SELECT synsetid FROM wordsXsensesXsynsets WHERE lemma = 'option') AND lemma <> 'option' ORDER BY synsetid
However, they never complete. At least not in any reasonable amount of time and I have had to cancel all of the queries. All other queries seem to work find and when I break up the second SQL example, I can get the individual parts to work and complete in reasonable times (about 0.40 seconds)
When I try and run the full statement however, the MySQL command line client just hangs.
Is there a problem with this syntax? What is causing it to take so long?
EDIT:
Output of "EXPLAIN SELECT ..."
Output of "EXPLAIN EXTENDED ...; SHOW WARNINGS;"
I did more digging and looking into the various statements used and found the problem was in the IN command.
MySQL repeats the statement for every single row in the database. This is the cause of the hang, as it had to run through hundreds of thousands of records.
My remedy to this was to split the command into two separate database calls first getting the synsets, and then dynamically creating a bound SQL string to look for the words in the synsets.
I have a string returned from a function "'aa','bb','cc',..."(the function uses GROUP_CONCAT). I want to use this as a condition in the IN clase of mysql.
SELECT name,class,comment
FROM vwstudent v
WHERE name IN (select func())
I want the query to act like
SELECT name,class,comment
FROM vwstudent v
WHERE name IN ('aa','bb','cc','dd',..)
I assume ('aa','bb','cc','dd',..) is acting as a whole string here and isn't generating any result. What can I do to run this query error less.
I'm not sure if it'll help. But you might be able to hack something together by using PREPARE and EXECUTE as described here: How To have Dynamic SQL in MySQL Stored Procedure
... but that's completely new to me. I've never used anything like that in MySQL.