I am receiving a list as input for a stored procedure in MySQL 5.6 and need to create a temporary table that has a a column (listOfUsers). Each item in the list, needs to be its own row for this column.
All the answers I've seen so far, show the list being used in a WHERE clause to filter a query. I am not trying to filter anything, just create a table with one column from a list.
Is this possible?
I am going to assume some specifics in the example, I hope my assumptions are relevant to your problem. Suppose we have a comma-delimited list on one line in a file. Then we can do:
create table t1 (s varchar(50));
load data local infile '/tmp/file.txt' into table t1
lines terminated by ',';
update t1 set s = replace(s, "\n", "");
The last update is needed to remove the spurious newline from the last column. Something based on this idea will hopefully solve your problem.
If the data is not coming from a file, a simple solution that requires minimum application coding is to put the data in a temporary file, and then apply the method above. Alternatively, you can just parse out the input in the application, and then build a multi-row insert.
I have the following table :
id x y z
1 z
3
6 x
7 zy
....
10000
I need to add id's in between the other id's that are already without deleting the data inside. Can't seem to find any solution, tryed all sorts of things but ended up making blank rows.
Kinda new to sql all together.
Using the information you provided in the comment...
Got a backup that i need to import to current db which is made with update only querys and need to create the rows first so that I can import them.
... and the fact that you are using MySQL, I think there is a simple solution for your problem.
Create a copy of your backup file (to have the original in case it doesn't work as expected), open it in a text editor and replace UPDATE <table_name> with INSERT INTO <table_name> (put the actual name of your table instead of <table_name>).
If some of the rows you want to import already exists in the table you have the following options to solve the conflicts:
use INSERT IGNORE INTO <table_name> as the replacement string to ignore the rows from the backup (the rows already existing in the table remain unmodified); technically, IGNORE doesn't ignore the rows you want to insert; it attempts to insert them and fails because they already exists but treats the failures as warnings (they are normally errors);
use REPLACE INTO <table_name> as the replacement string to replace the existing rows with the data from the backup; technically, REPLACE does DELETE followed by INSERT; it is not the best solution if the rows you want to insert are not complete.
Following is the code to bulk load data from a text file.
LOAD DATA LOCAL INFILE 'C:\\file.txt'
INTO TABLE datatable;
I have a table with two columns, an attribute and id, the primary key with the AUTO_INCREMENT index. Values for the attribute are given (one line for each row) in the text file.
I want the id (indexed "AUTO_INCREMENT) to be inserted itself and then increment itself. I think it is possible, but what will be the way to do it?
Try this one:
LOAD DATA LOCAL INFILE 'C:\\file.txt'
INTO TABLE datatable(`attribute`);
If this won't work, a table structure and a sample rows of your file.txt would help.
You could raw import everything from .txt into the Database (with your given command), so you have just the attributes there and then afterwards add the ID field later.
ALTER TABLE datatable ADD `id` MEDIUMINT NOT NULL AUTO_INCREMENT KEY
For detail explanation there is already a question for that:
Add a column to existing table and uniquely number them
I have a members table. Half the data/fields are populated through an online CMS.
But for the member's core contact detail fields, they come from a CSV exported from a desktop database.
I wanted to be able to upload this CSV and use the LOAD DATA command to update the members contact detail fields (matching on id) but without touching/erasing the other fields.
Is there a way to do this or must I instead loop through each row of the CSV and UPDATE... (if that's the case, any tips for the best way to do it?)
The Load Data Infile command supports the REPLACE keyword. This might be what you're looking for. From the manual:
REPLACE works exactly like INSERT,
except that if an old row in the table
has the same value as a new row for a
PRIMARY KEY or a UNIQUE index, the old
row is deleted before the new row is
inserted
The Load Data Infile command also has options where you can specify which columns to update, so perhaps you can upload the data, only specifying the columns which you want to update.
I have one table spread across two servers running MySql 4. I need to merge these into one server for our test environment.
These tables literally have millions of records each, and the reason they are on two servers is because of how huge they are. Any altering and paging of the tables will give us too huge of a performance hit.
Because they are on a production environment, it is impossible for me to alter them in any way on their existing servers.
The issue is the primary key is a unique auto incrementing field, so there are intersections.
I've been trying to figure out how to use the mysqldump command to ignore certain fields, but the --disable-keys merely alters the table, instead of getting rid of the keys completely.
At this point it's looking like I'm going to need to modify the database structure to utilize a checksum or hash for the primary key as a combination of the two unique fields that actually should be unique... I really don't want to do this.
Help!
To solve this problem, I looked up this question, found #pumpkinthehead's answer, and realized that all we need to do is find+replace the primary key in each row with the NULL so that mysql will use the default auto_increment value instead.
(your complete mysqldump command) | sed -e "s/([0-9]*,/(NULL,/gi" > my_dump_with_no_primary_keys.sql
Original output:
INSERT INTO `core_config_data` VALUES
(2735,'default',0,'productupdates/configuration/sender_email_identity','general'),
(2736,'default',0,'productupdates/configuration/unsubscribe','1'),
Transformed Output:
INSERT INTO `core_config_data` VALUES
(NULL,'default',0,'productupdates/configuration/sender_email_identity','general'),
(NULL,'default',0,'productupdates/configuration/unsubscribe','1'),
Note: This is still a hack; For example, it will fail if your auto-increment column is not the first column, but solves my problem 99% of the time.
if you don't care what the value of the auto_increment column will be, then just load the first file, rename the table, then recreate the table and load the second file. finally, use
INSERT newly_created_table_name (all, columns, except, the, auto_increment, column)
SELECT all, columns, except, the, auto_increment, column
FROM renamed_table_name
You can create a view of the table without the primary key column, then run mysqldump on that view.
So if your table "users" has the columns: id, name, email
> CREATE VIEW myView AS
SELECT name, email FROM users
Edit: ah I see, I'm not sure if there's any other way then.
Clone Your table
Drop the column in clone table
Dump the clone table without the structure (but with -c option to get complete inserts)
Import where You want
This is a total pain. I get around this issue by running something like
sed -e "s/([0-9]*,/(/gi" export.sql > expor2.sql
on the dump to get rid of the primary keys and then
sed -e "s/VALUES/(col1,col2,...etc.) VALUES/gi" LinxImport2.sql > LinxImport3.sql
for all of the columns except for the primary key. Of course, you'll have to be careful that ([0-9]*, doesn't replace anything that you actually want.
Hope that helps someone.
SELECT null as fake_pk, `col_2`, `col_3`, `col_4` INTO OUTFILE 'your_file'
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
FROM your_table;
LOAD DATA INFILE 'your_file' INTO TABLE your_table
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n';
For added fanciness, you can set a before insert trigger on your receiving table that sets the new primary key for reach row before the insertion occurs, thereby using regular dumps and still clearing your pk. Not tested, but feeling pretty confident about it.
Use a dummy temporary primary key:
Use mysqldump normally --opts -c. For example, your primary key is 'id'.
Edit the output files and add a row "dummy_id" to the structure of your table with the same type as 'id' (but not primary key of course). Then modify the INSERT statement and replace 'id' by 'dummy_id'. Once imported, drop the column 'dummy_id'.
jimyi was on the right track.
This is one of the reasons why autoincrement keys are a PITA. One solution is not to delete data but add to it.
CREATE VIEW myView AS
SELECT id*10+$x, name, email FROM users
(where $x is a single digit uniquely identifying the original database) either creating the view on the source database (which you hint may not be possible) or use an extract routine like that described by Autocracy or load the data into staging tables on the test box.
Alternatively, don't create the table on the test system - instead put in separate tables for the src data then create a view which fetches from them both:
CREATE VIEW users AS
(SELECT * FROM users_on_a) UNION (SELECT * FROM users_on_b)
C.
The solution I've been using is to just do a regular SQL export of the data I'm exporting, then removing the primary key from the insert statements using a RegEx find&replace editor. Personally I use Sublime Text, but I'm sure TextMate, Notepad++ etc. can do the same.
Then I just run the query in which ever database the data should be inserted to by copy pasting the query into HeidiSQL's query window or PHPMyAdmin. If there's a LOT of data I save the insert query to an SQL file and use file import instead. Copy & paste with huge amounts of text often makes Chrome freeze.
This might sound like a lot of work, but I rarely use more than a couple of minutes between the export and the import. Probably a lot less than I would use on the accepted solution. I've used this solution method on several hundred thousand rows without issue, but I think it would get problematic when you reach the millions.
I like the temporary table route.
create temporary table my_table_copy
select * from my_table;
alter table my_table_copy drop id;
// Use your favorite dumping method for the temporary table
Like the others, this isn't a one-size-fits-all solution (especially given OP's millions of rows) but even at 10^6 rows it takes several seconds to run but works.