Why is a temporary table allowed to be referenced only once? - mysql

Joining multiple times to a temporary table. Getting this error:
Error Code: 1137. Can't reopen table: 'a'
I googled it and found that there is a restriction of some kind on referencing the same temporary table multiple times in a query. Can anyone explain why this restriction exists?
Here is an exmaple of simple query that will cause this error:
CREATE TEMPORARY TABLE foo
SELECT * FROM shopify_us limit 10;
SELECT *
FROM (
SELECT *
FROM shopify_us
LIMIT 10
) boo
LEFT JOIN foo a ON a.customer_id = boo.customer_id
LEFT JOIN foo b on b.customer_id = boo.customer_id
However, if I simply remove the 2nd join, I no longer encounter the error.

The restriction is that you can't reference a temporary table multiple times in the same query. For example, doing a self-join on a temporary table cannot be done. But you can use a temporary table by multiple subsequent queries.
Here's the comment in the MySQL Server code that explains the reason, right before it raises this error:
/*
We're trying to use the same temporary table twice in a query.
Right now we don't support this because a temporary table is always
represented by only one TABLE object in THD, and it can not be
cloned. Emit an error for an unsupported behaviour.
*/
https://github.com/mysql/mysql-server/blob/8.0/sql/sql_base.cc#L7284-L7289
THD refers to the data structure for session-scoped information. Temporary tables are scoped to the session in which they are created. They aren't references to tables that can be shared by multiple sessions.
In other words, it's just a limitation in the code with respect to the data structure that tracks temporary tables.
See also the history of this as a bug report dating back to 2005: https://bugs.mysql.com/bug.php?id=10327
There is a workaround in MySQL 8.0: you can us a common table expression instead, for some kinds of uses for which you would have used a temp table.

Related

CREATE TABLE IF NOT EXISTS x AS (SELECT ...) inserts double records when executed twice?

I try to create a table as a select result of query as the following:
CREATE TABLE IF NOT EXISTS x AS (SELECT ...)
That works fine and a new table x is created with adequate columns and content.
But when I re-execute the ddl query the existing table is populated again with the same content (so double...)
After first execution, table x:
1 a
2 b
3 c
After secondexecution, table x:
1 a
2 b
3 c
1 a
2 b
3 c
Why? And how to resolva that? because this executed by a user in my app so it can be executed many times and that's why I put IF NOT EXISTS to not re create the table each time...
Best Regards,
This is version dependent. As described in the documentation:
For CREATE TABLE ... SELECT, if IF NOT EXISTS is given and the
destination table already exists, the result is version dependent.
Before MySQL 5.5.6, MySQL handles the statement as follows:
The table definition given in the CREATE TABLE part is ignored. No error occurs, even if the definition does not match that of the
existing table. MySQL attempts to insert the rows from the SELECT part
anyway.
MySQL applies these rules when various CREATE ... IF NOT EXISTS statements are replicated:
Every CREATE DATABASE IF NOT EXISTS statement is replicated, whether
or not the database already exists on the master.
Similarly, every CREATE TABLE IF NOT EXISTS statement without a SELECT
is replicated, whether or not the table already exists on the master.
This includes CREATE TABLE IF NOT EXISTS ... LIKE. Replication of
CREATE TABLE IF NOT EXISTS ... SELECT follows somewhat different
rules; see Section 17.4.1.6, “Replication of CREATE TABLE ... SELECT
Statements”, for more information.
For more information- CREATE EVENT IF NOT EXISTS is always replicated in MySQL 5.5, whether
or not the event named in the statement already exists on the master.

SQL Server tables : It takes a long time to join temporary table variable ( #table ) in SQL Server 2014

I create a temporary table variable (#table). Then I inner join it with another table. It takes a long time to display result. I try again with #table. It's normal. What's wrong with it?
If you store too much data in temp table or temp table variable then it result in poor performance. Because these temp tables variables not allow indexing and in temp table most of the time developer forget to use proper indexing therefore during join complete table scan happen which slow the query output.
Another important point to notice, avoid joins on varchar column.

MySQL create table with STORAGE MEMORY clause

I encountered the following problem:
I have temporary table
CREATE temporary TABLE IF NOT EXISTS tmp_GL_VAR
(
G_TABLE_NAME VARCHAR(100) DEFAULT '',
G_DATE DATETIME,
G_ERROR_CODE INT DEFAULT 0
);
And I need to use it several times in single queries,for instance,
update t1 set c1 = (select G_TABLE_NAME from tmp_GL_VAR), c2 = (select G_ERROR_CODE from tmp_GL_VAR);
in functions with cursors, etc. However, in all these cases MySQL throws error:
SQL Error(1137): Can't reopen table 'tmp_GL_VAR'.
Then I tried to create permanent table with STORAGE MEMORY clause (also tried ENGINE MEMORY clause), hoping that table will be cleaned when the session ends
CREATE TABLE GL_VAR
(
G_TABLE_NAME VARCHAR(100) DEFAULT '',
G_DATE DATETIME,
G_ERROR_CODE INT DEFAULT 0
) STORAGE MEMORY;
But unfortunately this option had no effect. The data was available across different sessions (connections).
Please advise how I can bypass 'Can't reopen table' without rewriting all queries, stored functions, etc. (there're far too many LOC).
The requirement is: table should be either dropped or at least truncated as session ends and the data from one session shouldn't be available in another session (each user can see only its own data in this table).
Any help is appreciated.
Yes. there are problems http://dev.mysql.com/doc/refman/5.1/en/temporary-table-problems.html
You cannot refer to a TEMPORARY table more than once in the same query. For example, the following does not work:
mysql> SELECT * FROM temp_table, temp_table AS t2;
ERROR 1137: Can't reopen table: 'temp_table'
This error also occurs if you refer to a temporary table multiple times in a stored function under different aliases, even if the references occur in different statements within the function.
I think u should use usual table for this purpose.
And add some session identifier to make it work in your multi-user system. (Yes, u will have to add this identifier to ALL other queries using this table)
Delete rows by this identifier (or old timestamps if you want) any time u wish

How can I store the output of a query into a temporary table and use the table in a new query?

I have a MySQL query which uses 3 tables with 2 inner joins. Then, I have to find the maximum of a group from this query output. Combining them both is beyond me. Can I break down the problem by storing the output of the first complicated query into some sort of temporary table, give it a name and then use this table in a new query? This will make the code more manageable. Thank you for your help.
This is very straightforward:
CREATE TEMPORARY TABLE tempname AS (
SELECT whatever, whatever
FROM rawtable
JOIN othertable ON this = that
)
The temporary table will vanish when your connection closes. A temp table contains the data that was captured at the time it was created.
You can also create a view, like so.
CREATE VIEW viewname AS (
SELECT whatever, whatever
FROM rawtable
JOIN othertable ON this = that
)
Views are permanent objects (they don't vanish when your connection closes) but they retrieve data from the underlying tables at the time you invoke them.

Raising mySQL 61 table join limit

I have a Drupal 6 application that requires more joins than that 61 table join mySQL limit allows. I understand that this is an excessive number, but it is ran only once a day, and the results are cached for further reference.
Are there any mySQL configuration parameters that could be of help, or any other approaches short of changing the logic behind collecting the data?
My approach would be to split the humongous query into smaller, simpler queries, and use temporary tables to store the intermediate steps. I use this approach frequently and it helps me a lot (sometimes it is even faster to create some temp tables than to join all the tables in one big query).
Something like this:
drop table if exists temp_step01;
create temporary table temp_step01
select t1.*, t2.someField
from table1 as t1 inner join table2 as t2 on t1.id = t2.table1_id;
-- Add the appropriate indexes to optimize the subsequent queries
alter table temp_step01
add index idx_1 (field1);
-- Create all the temp tables that you need, and finally show the results
select sXX.*
from temp_stepXX as sXX;
Remember: Temporary tables are visible only to the connection that creates them. If you need to make the result visible to other connections, you'll need to create a "real" table (of course, that is only worth with the last step of your process).