I'm currently dumping MySQL tables with mysqldump.
The dump is currently producing:
DROP TABLE IF EXISTS `versions`;
/*!40101 SET #saved_cs_client = ##character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `versions` (
`major` int(11) NOT NULL DEFAULT '0',
`minor` int(11) NOT NULL DEFAULT '0',
`revision` int(11) NOT NULL DEFAULT '0',
`build` int(11) NOT NULL DEFAULT '0',
`date_installed` datetime NOT NULL,
`current` tinyint(4) NOT NULL DEFAULT '0',
`product_type` varchar(30) DEFAULT NULL,
`product` varchar(30) DEFAULT NULL,
`product_class_name` varchar(80) DEFAULT NULL,
`lazy_load` tinyint(4) NOT NULL DEFAULT '0',
`sitewide` tinyint(4) NOT NULL DEFAULT '0',
UNIQUE KEY `versions_pkey` (`product`,`major`,`minor`,`revision`,`build`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
What I'd like the dump to say is something like,
IF TABLE EXISTS, UPDATE IT TO MATCH THIS STRUCTURE (
`major` int(11) NOT NULL DEFAULT '0',
`minor` int(11) NOT NULL DEFAULT '0',
`revision` int(11) NOT NULL DEFAULT '0',
`build` int(11) NOT NULL DEFAULT '0',
`date_installed` datetime NOT NULL,
`current` tinyint(4) NOT NULL DEFAULT '0',
`product_type` varchar(30) DEFAULT NULL,
`product` varchar(30) DEFAULT NULL,
`product_class_name` varchar(80) DEFAULT NULL,
`lazy_load` tinyint(4) NOT NULL DEFAULT '0',
`sitewide` tinyint(4) NOT NULL DEFAULT '0',
UNIQUE KEY `versions_pkey` (`product`,`major`,`minor`,`revision`,`build`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Is this possible through MySQL dump (or indeed any other MySQL-related tools)?
Thanks
You may need to dump the schema ahead of time and convert CREATE TABLE to CREATE TABLE IF NOT EXISTS
mysqldump -u... -p... --no-data --skip-add-drop-table ... --all-databases | sed 's/CREATE TABLE/CREATE TABLE IF NOT EXISTS/g' > MYSQLSchema.sql
This will bypass adding the DROP TABLE IF EXISTS versions; command and transform CREATE TABLE into CREATE TABLE IF NOT EXISTS.
Give it a Try !!!
When you dump the database, you can set/reset the flag that generates the DROP TABLE command.
Related
SQL query:
CREATE TABLE `wp_layerslider` (
`id` int(10) NOT NULL AUTO_INCREMENT,
`author` int(10) NOT NULL DEFAULT '0',
`name` varchar(100) COLLATE utf8mb4_unicode_520_ci DEFAULT '',
`slug` varchar(100) COLLATE utf8mb4_unicode_520_ci DEFAULT '',
`data` mediumtext COLLATE utf8mb4_unicode_520_ci NOT NULL,
`date_c` int(10) NOT NULL,
`date_m` int(10) NOT NULL,
`schedule_start` int(10) NOT NULL DEFAULT '0',
`schedule_end` int(10) NOT NULL DEFAULT '0',
`flag_hidden` tinyint(1) NOT NULL DEFAULT '0',
`flag_deleted` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
MySQL said: Documentation
1273 - Unknown collation: 'utf8mb4_unicode_520_ci'
When attempting to import a database. I'm very new to all of this, hoping someone can assist in simple terms for me
could be you are using an oldeversion of mysql db .. and the collate you need is not avialable .. you could use try alter the table or change the collate
ALTER TABLE `wp_layerslider` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci
I have a big website, at the moment i have about 10k unique visitors daily, I have 2000 categories, 12k Tags, 36k custom taxonomies.
It's not a design flaw, as its a website to watch online movies, I used, categories for Seasons and stuff like that, the custom taxonomies are Actors, regizors etc.
The problem is that my hosting keeps crashing with 100% processor, the resources cannot be an issue, as i have my own server with 4xXeon, 64 GB ram.
I have isolated the problem, and it's due to very slow queries for example, what are my options? :
# Time: 160830 17:01:20
# User#Host: razvypp[razvypp] # localhost []
# Query_time: 9.159090 Lock_time: 0.001680 Rows_sent: 30 Rows_examined: 420117
SET timestamp=1472565680;
SELECT wphn_posts.ID
FROM wphn_posts
INNER JOIN wphn_term_relationships
ON (wphn_posts.ID = wphn_term_relationships.object_id)
WHERE 1=1
AND ( wphn_term_relationships.term_taxonomy_id IN (3630,
4955,4956,4957,4958,4959,4960,4961,4962,4963,4964,4965,
4966,4967,4968,4969,4970,4971,4972,4973,4974,4975,4976,
4977,4978,4979,4980,4981,4982,4983,4984,4985,4986,4987,
4988,4989,4990,4991,4992,4993,4994,4995,4996,4997,4998,
4999,5000,5001,5002,5003,5004,5005,5006,5007,5008,5009,
5010,5011,5012, ... (hundreds of numbers) ...,49740,49779)
Here is a dump of the tables
-- phpMyAdmin SQL Dump
-- version 4.4.15
-- http://www.phpmyadmin.net
--
-- Host: localhost
-- Generation Time: 30 Aug 2016 la 15:17
-- Versiune server: 5.5.51-log
-- PHP Version: 5.5.37
SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
SET time_zone = "+00:00";
/*!40101 SET #OLD_CHARACTER_SET_CLIENT=##CHARACTER_SET_CLIENT */;
/*!40101 SET #OLD_CHARACTER_SET_RESULTS=##CHARACTER_SET_RESULTS */;
/*!40101 SET #OLD_COLLATION_CONNECTION=##COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8mb4 */;
--
-- Database: `trolio`
--
-- --------------------------------------------------------
--
-- Structura de tabel pentru tabelul `wphn_posts`
--
CREATE TABLE IF NOT EXISTS `wphn_posts` (
`ID` bigint(20) unsigned NOT NULL,
`movie_type` int(2) NOT NULL,
`post_author` bigint(20) unsigned NOT NULL DEFAULT '0',
`post_date` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`post_date_gmt` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`post_content` longtext NOT NULL,
`post_title` text NOT NULL,
`post_excerpt` text NOT NULL,
`post_status` varchar(20) NOT NULL DEFAULT 'publish',
`comment_status` varchar(20) NOT NULL DEFAULT 'open',
`ping_status` varchar(20) NOT NULL DEFAULT 'open',
`post_password` varchar(20) NOT NULL DEFAULT '',
`post_name` varchar(200) NOT NULL DEFAULT '',
`to_ping` text NOT NULL,
`pinged` text NOT NULL,
`post_modified` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`post_modified_gmt` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`post_content_filtered` longtext NOT NULL,
`post_parent` int(20) unsigned NOT NULL DEFAULT '0',
`guid` varchar(255) NOT NULL DEFAULT '',
`menu_order` int(11) NOT NULL DEFAULT '0',
`post_type` varchar(20) NOT NULL DEFAULT 'post',
`post_mime_type` varchar(100) NOT NULL DEFAULT '',
`comment_count` bigint(20) NOT NULL DEFAULT '0',
`views` int(99) NOT NULL,
`views_this_week` int(5) NOT NULL,
`imdb` double NOT NULL,
`imdb_votes` float NOT NULL,
`imdb_id` varchar(20) NOT NULL,
`fb_shares_root` int(5) NOT NULL,
`movie_name` varchar(140) NOT NULL,
`season` int(2) NOT NULL,
`episode` int(5) NOT NULL,
`serial_id` int(10) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
-- --------------------------------------------------------
--
-- Structura de tabel pentru tabelul `wphn_term_relationships`
--
CREATE TABLE IF NOT EXISTS `wphn_term_relationships` (
`object_id` bigint(20) unsigned NOT NULL DEFAULT '0',
`term_taxonomy_id` bigint(20) unsigned NOT NULL DEFAULT '0',
`term_order` int(11) NOT NULL DEFAULT '0'
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
-- --------------------------------------------------------
--
-- Structura de tabel pentru tabelul `wphn_term_taxonomy`
--
CREATE TABLE IF NOT EXISTS `wphn_term_taxonomy` (
`term_taxonomy_id` bigint(20) unsigned NOT NULL,
`term_id` bigint(20) unsigned NOT NULL DEFAULT '0',
`taxonomy` varchar(32) NOT NULL DEFAULT '',
`description` longtext NOT NULL,
`parent` bigint(20) unsigned NOT NULL DEFAULT '0',
`count` bigint(20) NOT NULL DEFAULT '0'
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
--
-- Indexes for dumped tables
--
--
-- Indexes for table `wphn_posts`
--
ALTER TABLE `wphn_posts`
ADD PRIMARY KEY (`ID`),
ADD KEY `post_name` (`post_name`(191)),
ADD KEY `type_status_date` (`post_type`,`post_status`,`post_date`,`ID`),
ADD KEY `post_parent` (`post_parent`),
ADD KEY `post_author` (`post_author`),
ADD KEY `serial_id` (`serial_id`),
ADD KEY `episode` (`episode`),
ADD KEY `season` (`season`),
ADD KEY `movie_name` (`movie_name`);
--
-- Indexes for table `wphn_term_relationships`
--
ALTER TABLE `wphn_term_relationships`
ADD PRIMARY KEY (`object_id`,`term_taxonomy_id`),
ADD KEY `term_taxonomy_id` (`term_taxonomy_id`),
ADD KEY `term_order` (`term_order`);
--
-- Indexes for table `wphn_term_taxonomy`
--
ALTER TABLE `wphn_term_taxonomy`
ADD PRIMARY KEY (`term_taxonomy_id`),
ADD UNIQUE KEY `term_id_taxonomy` (`term_id`,`taxonomy`),
ADD KEY `taxonomy` (`taxonomy`),
ADD KEY `parent` (`parent`),
ADD KEY `count` (`count`),
ADD KEY `term_id` (`term_id`);
--
-- AUTO_INCREMENT for dumped tables
--
--
-- AUTO_INCREMENT for table `wphn_posts`
--
ALTER TABLE `wphn_posts`
MODIFY `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT;
--
-- AUTO_INCREMENT for table `wphn_term_taxonomy`
--
ALTER TABLE `wphn_term_taxonomy`
MODIFY `term_taxonomy_id` bigint(20) unsigned NOT NULL AUTO_INCREMENT;
/*!40101 SET CHARACTER_SET_CLIENT=#OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=#OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=#OLD_COLLATION_CONNECTION */;
"Rows examined" = 420K -- That smacks of a missing index.
Probably missing:
INDEX(term_taxonomy_id, object_id)
Please provide SHOW CREATE TABLE for the two tables.
What is the value of innodb_buffer_pool_size?
The problem that I am going to tell is for all the tables in the DB, When I dump my DB it ignores AUTO_INCREMENT attribute. My actual table is as the following:
CREATE TABLE IF NOT EXISTS `departments` (
`departmentid` int(11) NOT NULL AUTO_INCREMENT,
`chairid` int(11) NOT NULL,
`department_name` varchar(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`image` varchar(128) NOT NULL DEFAULT 'default_department.png',
`url` varchar(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`active` tinyint(4) NOT NULL DEFAULT '1'
) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=utf8;
But if I dump the table using PHPMyAdmin it does not add auto_increment which I mentioned before.
The output of the exported .sql file's content is here:
CREATE TABLE IF NOT EXISTS `departments` (
`departmentid` int(11) NOT NULL, -- AUTO_INCREMENT is missing
`chairid` int(11) NOT NULL,
`department_name` varchar(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`image` varchar(128) NOT NULL DEFAULT 'default_department.png',
`url` varchar(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`active` tinyint(4) NOT NULL DEFAULT '1'
) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=utf8;
I especially checked that if auto_increment is disabled in CREATE TABLE options but no, it is not.
This was fixed in phpMyAdmin version 4.5.0.1 (Sep 2015):
https://github.com/phpmyadmin/phpmyadmin/issues/11492
I supposed it ignores auto increment but I explore file and at the bottom of sql I found that it altered to auto increment somewhere else
When running mysqldump -d I get this format:
--
-- Table structure for table `wp_users`
--
DROP TABLE IF EXISTS `wp_users`;
/*!40101 SET #saved_cs_client = ##character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `wp_users` (
`ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`user_login` varchar(60) NOT NULL DEFAULT '',
`user_pass` varchar(64) NOT NULL DEFAULT '',
`user_nicename` varchar(50) NOT NULL DEFAULT '',
`user_email` varchar(100) NOT NULL DEFAULT '',
`user_url` varchar(100) NOT NULL DEFAULT '',
`user_registered` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`user_activation_key` varchar(60) NOT NULL DEFAULT '',
`user_status` int(11) NOT NULL DEFAULT '0',
`display_name` varchar(250) NOT NULL DEFAULT '',
PRIMARY KEY (`ID`),
KEY `user_login_key` (`user_login`),
KEY `user_nicename` (`user_nicename`)
) ENGINE=MyISAM AUTO_INCREMENT=722 DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = #saved_cs_client */;
/*!40103 SET TIME_ZONE=#OLD_TIME_ZONE */;
How do I get this format:
CREATE TABLE `wp_users` (
`ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`user_login` varchar(60) NOT NULL DEFAULT '',
`user_pass` varchar(64) NOT NULL DEFAULT '',
`user_nicename` varchar(50) NOT NULL DEFAULT '',
`user_email` varchar(100) NOT NULL DEFAULT '',
`user_url` varchar(100) NOT NULL DEFAULT '',
`user_registered` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`user_activation_key` varchar(60) NOT NULL DEFAULT '',
`user_status` int(11) NOT NULL DEFAULT '0',
`display_name` varchar(250) NOT NULL DEFAULT '',
PRIMARY KEY (`ID`),
KEY `user_login_key` (`user_login`),
KEY `user_nicename` (`user_nicename`)
)
I'm trying to compare database structures using diff, but keep getting these lines I have to grep out.
mysqldump -d --compact --compatible=mysql323 ${dbname}|egrep -v "(^SET|^/\*\!)" |sed "s/AUTO_INCREMENT=[0-9]*//g"
Is it possible for mysqldump to dump one query per line?
For example, it currently dumps a CREATE TABLE expressions like so:
--
-- Table structure for table `post`
--
CREATE TABLE IF NOT EXISTS `post` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`title` varchar(160) NOT NULL,
`slug` varchar(255) NOT NULL,
`url` varchar(600) NOT NULL,
`domain` varchar(90) NOT NULL,
`author` int(11) NOT NULL,
`description` text,
`category` int(11) DEFAULT NULL,
`score` int(11) NOT NULL,
`ip` varchar(255) DEFAULT NULL,
`created` datetime NOT NULL,
`comment_count` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `uc_slug` (`slug`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1;
-- --------------------------------------------------------
--
-- Table structure for table `users`
--
CREATE TABLE IF NOT EXISTS `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`login` varchar(12) NOT NULL,
`password` varchar(45) NOT NULL,
`email` varchar(150) NOT NULL,
`about` varchar(300) DEFAULT NULL,
`last_visit` datetime DEFAULT NULL,
`ip` varchar(255) DEFAULT NULL,
`created` datetime NOT NULL,
`perm_mod` int(11) DEFAULT NULL,
`perm_admin` int(11) DEFAULT NULL,
`post_count` int(11) NOT NULL DEFAULT '0',
`comment_count` int(11) NOT NULL DEFAULT '0',
`vote_count` int(11) NOT NULL DEFAULT '0',
`voted_count` int(11) NOT NULL DEFAULT '0',
`forgot_key` varchar(150) DEFAULT NULL,
`cookie_key` varchar(40) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=3;
Instead, I want it to dump like this:
--
-- Table structure for table `post`
--
CREATE TABLE IF NOT EXISTS `post` (`id` int(11) NOT NULL AUTO_INCREMENT,`title` varchar(160) NOT NULL,`slug` varchar(255) NOT NULL,`url` varchar(600) NOT NULL,`domain` varchar(90) NOT NULL,`author` int(11) NOT NULL,`description` text,`category` int(11) DEFAULT NULL,`score` int(11) NOT NULL,`ip` varchar(255) DEFAULT NULL,`created` datetime NOT NULL,`comment_count` int(11) NOT NULL DEFAULT '0',PRIMARY KEY (`id`),UNIQUE KEY `uc_slug` (`slug`) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1;
-- --------------------------------------------------------
--
-- Table structure for table `users`
--
CREATE TABLE IF NOT EXISTS `users` (`id` int(11) NOT NULL AUTO_INCREMENT,`login` varchar(12) NOT NULL,`password` varchar(45) NOT NULL,`email` varchar(150) NOT NULL,`about` varchar(300) DEFAULT NULL,`last_visit` datetime DEFAULT NULL,`ip` varchar(255) DEFAULT NULL,`created` datetime NOT NULL,`perm_mod` int(11) DEFAULT NULL,`perm_admin` int(11) DEFAULT NULL,`post_count` int(11) NOT NULL DEFAULT '0',`comment_count` int(11) NOT NULL DEFAULT '0',`vote_count` int(11) NOT NULL DEFAULT '0',`voted_count` int(11) NOT NULL DEFAULT '0',`forgot_key` varchar(150) DEFAULT NULL,`cookie_key` varchar(40) DEFAULT NULL,PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=3;
I have been going through the args for mysqldump, and can't find anything that would do this
Indeed, there is no command-line switch to achieve this. Instead I recommend you to do the dump normally and then use a good text editor like Notepad++ to open dump file.
First select block to convert to single line, then select menu option of the folowing image.
This is the result (repeat for every block to convert to single line):
I had similar task - to format mysqldump output file to be mysql init_file option compatible. Here is the solution:
sed ':a;N;$!ba;s/\n/THISISUNIQUESTRING/g' | sed -e 's/;THISISUNIQUESTRING/;\n/g' | sed -e 's/THISISUNIQUESTRING//g'
This replaces all newline chars with a weird string that is not present in the file. Next sed replaces the combination of this string and semicolon with semicolon and newline char. This is needed because table data can contain string values that include ";", so straighforward removal of all newline chars and adding them after semicolons is not universal.
After that remaining weird strings are just removed from the text.
Btw, I was running mysqldump with --comments=false parameter, as comments are not allowed for "init_file" thing as well.
If you truly want individual queries the most pragmatic solution in my book would be to use the built-in functionality of mysqldump. The --skip-extended-insert flag is what you want, it is not clear in the documentation but it gives you the dump format you want.