CREATE TABLE as SELECT - using MEMORY ENGINE (in RAM memory) - mysql

I was reading about tables using MEMORY ENGINE (tables stored in ram).
Is it possible to use CREATE TABLE AS SELECT syntax, but have the created table us MEMORY ENGINE?
Is there a limit to the size of a MEMORY table? Could create a MEMORY table that is a copy of a pysical 1.5 GB table?

Yes, it can be done, but not with simple syntax CREATE TABLE t AS SELECT ... - you will need to specify table columns in full statement, Sample:
CREATE TABLE t (col1 INT(11), col2 INT(11))
ENGINE=MEMORY
AS
SELECT * FROM another_t
Maximum size of table by default if 16Mb, but it can be adjusted with max_heap_table_size server system variable. But please, note, that this limit is per engine - not per table. I.e. all your memory tables will share it. To restrict size of individual table, you'll need to operate on session value for max_heap_table_size, like
mysql> SET max_heap_table_size = 24*1024*1024;

you can also create a temporary table this way:
CREATE TEMPORARY TABLE IF NOT EXISTS tmp ENGINE=MEMORY AS (
SELECT 'abc' As tomcat, 'def' As apache);
Or this way:
CREATE TEMPORARY TABLE IF NOT EXISTS tmp ENGINE=MEMORY AS (
SELECT tomcat, apache From servers);
Be advised, the first option will create the two columns as varchar(3), meanwhile the second option will import any column information from the original table, even their comments.

Related

MySQL create table with STORAGE MEMORY clause

I encountered the following problem:
I have temporary table
CREATE temporary TABLE IF NOT EXISTS tmp_GL_VAR
(
G_TABLE_NAME VARCHAR(100) DEFAULT '',
G_DATE DATETIME,
G_ERROR_CODE INT DEFAULT 0
);
And I need to use it several times in single queries,for instance,
update t1 set c1 = (select G_TABLE_NAME from tmp_GL_VAR), c2 = (select G_ERROR_CODE from tmp_GL_VAR);
in functions with cursors, etc. However, in all these cases MySQL throws error:
SQL Error(1137): Can't reopen table 'tmp_GL_VAR'.
Then I tried to create permanent table with STORAGE MEMORY clause (also tried ENGINE MEMORY clause), hoping that table will be cleaned when the session ends
CREATE TABLE GL_VAR
(
G_TABLE_NAME VARCHAR(100) DEFAULT '',
G_DATE DATETIME,
G_ERROR_CODE INT DEFAULT 0
) STORAGE MEMORY;
But unfortunately this option had no effect. The data was available across different sessions (connections).
Please advise how I can bypass 'Can't reopen table' without rewriting all queries, stored functions, etc. (there're far too many LOC).
The requirement is: table should be either dropped or at least truncated as session ends and the data from one session shouldn't be available in another session (each user can see only its own data in this table).
Any help is appreciated.
Yes. there are problems http://dev.mysql.com/doc/refman/5.1/en/temporary-table-problems.html
You cannot refer to a TEMPORARY table more than once in the same query. For example, the following does not work:
mysql> SELECT * FROM temp_table, temp_table AS t2;
ERROR 1137: Can't reopen table: 'temp_table'
This error also occurs if you refer to a temporary table multiple times in a stored function under different aliases, even if the references occur in different statements within the function.
I think u should use usual table for this purpose.
And add some session identifier to make it work in your multi-user system. (Yes, u will have to add this identifier to ALL other queries using this table)
Delete rows by this identifier (or old timestamps if you want) any time u wish

ERROR 1878 (HY000): Temporary file write failure

I am executing a query
ALTER TABLE message ADD COLUMN syncid int(10) NOT NULL DEFAULT 0;
MySQL returned error:
ERROR 1878 (HY000): Temporary file write failure.
message table info:
engine type:InnoDB
rows:15786772
index length:1006.89 MB
data length:11.25 GB
How to fix it?
MySQL implements ALTER TABLE as a table re-creation, so two copies of the table exists on the system at some stage during the process. You will need over 12 GB free space for this operation.
Free some space. Alternatively, set your server to use a different temporary directory, where there is enough space.
Alternative to the alternative (the WHILE might need to be wrapped in a stored procedure):
create a new table (temp_table) with the new structure
transfer data in small batches from original_table into temp_table
drop original_table and rename temp_table
-- useful only if concurrent access is allowed during migration
LOCK TABLES original_table WRITE, temp_table WRITE;
SELECT COUNT(*) INTO #anythingleft FROM original_table;
WHILE #anythingleft DO
-- transfer data
INSERT INTO temp_table
SELECT
original_table.old_stuff,
"new stuff"
FROM original_table
ORDER BY any_sortable_column_with_unique_constraint -- very important!
LIMIT 1000; -- batch size, adjust to your situation
DELETE FROM original_table
ORDER BY any_sortable_column_with_unique_constraint
LIMIT 1000; -- ORDER BY and LIMIT clauses MUST be exactly the same as above
SELECT COUNT(*) INTO #anythingleft FROM original_table;
END WHILE;
-- delete, rename
DROP TABLE original_table;
UNLOCK TABLES;
RENAME TABLE old_table TO original_table;
If your table uses InnoDB, a more elaborate solution is possible with SELECT ... FOR UPDATE; instead of table locks, but I trust you get the idea.
Sorry for the late answer or digging up this old topic, but the following tools can help you with that:
pt-online-schema-change
github/gh-ost
Both tools recreate the table in the fashion that #RandomSeed proposed, but in a simpler way.
However please ensure that there is enough space on the file system. Those tools don't need more space in the temporary folder, which is interesting when you're mounting your temporary folder on a separate drive / RAMdisk.

TEMPORARY table in Mysql

If two users create two TEMPORARY tables at the same time in my mysql database using a PHP script, will it create two different tables? Can those users use their own table without facing any trouble and will those table be automatically deleted?
thanks :)
CREATE TEMPORARY TABLE TempTable ( ID int, Name char(100) ) TYPE=HEAP;
INSERT INTO TempTable VALUES( 1, "Foo bar" );
SELECT * FROM TempTable;
DROP TABLE TempTable;
As stated in the manual:
A TEMPORARY table is visible only to the current connection, and is dropped automatically when the connection is closed. This means that two different connections can use the same temporary table name without conflicting with each other or with an existing non-TEMPORARY table of the same name. (The existing table is hidden until the temporary table is dropped.)
Temporary tables are created in a current session; so, two users can create two temp. tables at the same time in their threads. These tables will be removed on session closing.
You can use [IF NOT EXISTS] in this query...
reference

Create a temporary table in a SELECT statement without a separate CREATE TABLE

Is it possible to create a temporary (session only) table from a select statement without using a create table statement and specifying each column type? I know derived tables are capable of this, but those are super-temporary (statement-only) and I want to re-use.
It would save time if I did not have to write up a create table command and keep the column list and type list matched up.
CREATE TEMPORARY TABLE IF NOT EXISTS table2 AS (SELECT * FROM table1)
From the manual found at http://dev.mysql.com/doc/refman/5.7/en/create-table.html
You can use the TEMPORARY keyword when creating a table. A TEMPORARY table is visible only to the current session, and is dropped automatically when the session is closed. This means that two different sessions can use the same temporary table name without conflicting with each other or with an existing non-TEMPORARY table of the same name. (The existing table is hidden until the temporary table is dropped.) To create temporary tables, you must have the CREATE TEMPORARY TABLES privilege.
In addition to psparrow's answer if you need to add an index to your temporary table do:
CREATE TEMPORARY TABLE IF NOT EXISTS
temp_table ( INDEX(col_2) )
ENGINE=MyISAM
AS (
SELECT col_1, coll_2, coll_3
FROM mytable
)
It also works with PRIMARY KEY
Use this syntax:
CREATE TEMPORARY TABLE t1 (select * from t2);
Engine must be before select:
CREATE TEMPORARY TABLE temp1 ENGINE=MEMORY
as (select * from table1)
ENGINE=MEMORY is not supported when table contains BLOB/TEXT columns
As I understand it, a SELECT statement will work on the temporary table if you're using it in something like phpMyAdmin, but following that SELECT, the temporary table will be gone. This means set up exactly what you want to do with it first, and don't view any results till your 'action' statements that change the data (DELETE, UPDATE) are complete.

MySQL temporary vs memory table in stored procedures

What's is better to use in a stored procedure: a temporary table or a memory table?
The table is used to stored summary data for reports.
Are there any trade offs that developers should be aware off?
CREATE TEMPORARY TABLE t (avg (double));
or
CREATE TABLE t (avg (double)) ENGINE=MEMORY;
Why is this restricted to just the two options? You can do:
CREATE TEMPORARY TABLE t (avg double) ENGINE=MEMORY;
Which works, although I'm not sure how to check if the memory engine is actually being used here.
Of the two, I'd use a temporary table for report.
A memory table holds data across user sessions & connections, so you'd have to truncate it every time to make sure you wouldn't be using data from someone else. Assuming you put in whats necessary to maintain a memory table depending on your needs, it's fine - the temp table is a little safer from a maintenance perspective.
A temporary table will only exist for the duration of your session. A table declared with Engine=Memory will persist across user sessions / connections but will only exist in the lifetime of the MySQL instance. So if MySQL gets restarted the table goes away.
In MySQL, temporary tables are seriously crippled:
http://dev.mysql.com/doc/refman/5.6/en/temporary-table-problems.html
You cannot refer to a TEMPORARY table more than once in the same query.
For example, the following does not work:
mysql> SELECT * FROM temp_table, temp_table AS t2;
ERROR 1137: Can't reopen table: 'temp_table'
I Just wanted to point out that, in 2021 using MariaDB-10.3.27, the code #biziclop said doesn't work, is not the case any more, this is possible:
CREATE TEMPORARY TABLE tmp1 AS
SELECT * FROM products LIMIT 10;
SELECT * FROM tmp1, tmp1 AS t2;
(I just tested it)