Executing Multiple Statements Using Knex & MySQL (Variables & Migration) - mysql

Question

Problem:
I'm trying to add a new column to a table with tens of millions of rows, without blocking it.
I'm aware there are a few options here, and I've given ALTER TABLE a try with NULL values & also with a default value, but it takes a very long time (not workable) and locks the table.
So I'm trying to write a migration script that will duplicate the structure of the original table to a new table, add my new column to that new table, and then slowly migrate the old table data in to the new table.
My issue is with Knex though on the data copy.
I'm testing locally and the below query runs absolutely fine in MySQL version 5.6.34 in Sequel Pro, but I get a You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' in Knex.
knex.schema.raw(`
SELECT #prevMaxIdMovedRecord;
SELECT #newMaxIdMovedRecord;
SELECT maxIdMoved FROM migration_status
ORDER BY maxIdMoved DESC
INTO #prevMaxIdMovedRecord;
INSERT IGNORE INTO table_copy (field1, newField1)
SELECT t.field1, 1 FROM table t
WHERE t.id BETWEEN #prevMaxIdMovedRecord AND #prevMaxIdMovedRecord + 50000;
SELECT id FROM table_copy
ORDER BY id DESC
LIMIT 1
INTO #newMaxIdMovedRecord;
INSERT INTO migration_status (maxIdMoved)
VALUES (#newMaxIdMovedRecord);
DELIMITER ;
`)
Any experts here that can assist? I can't see that Knex allows you to specify MySQL version in the Knexfile.js, and I was wondering if maybe there was a mismatch between versions on syntax. All my other commands in terms of new tables etc work fine...
Thanks

Answer 1

The mysql driver won't accept multiple statements in a single query by default. You can demonstrate this with a simple Node program:
const mysql = require('mysql')
connection = mysql.createConnection({
host: 'localhost',
user: 'youruser',
password: 'yourpassword',
database: 'yourdb',
multipleStatements: true
})
connection.connect()
connection.query('SELECT 1+1; SELECT 2+2;', function(err, results) {
if (err) {
throw err
}
console.log(results[0], results[1])
})
connection.end()
If you set multipleStatements to false, you should see an error. While multipleStatements could be passed to the driver via your knexfile.js,
{
client: "mysql",
connection: {
// ... other options ...
multipleStatements: true
}
}
and you might try this, I recommend you use a transaction as executing multiple statements in one query like this on a large table seems fragile at best. See docs for an introduction.