Converting multiple lines of mysql string into predefined csv format? - mysql

this is more like a how-to question about csv, excel, mysql files than a standard coding question. However I really would appreciat some help with this.
I have this predefined CSV format, that I need in order to import it to a database with Sequel Pro.
ID,"user_login","user_pass","user_nicename","user_email","user_url","user_registered","user_activation_key",user_status,"display_name"
1,"some_name","$P$BDMmGci3KAqNQwfPtgzfQTp8D3k7bC.","some_name","some.name#gmx.net","","2013-06-27 11:32:08","",0,"Some Name"
On the other hand I have an old database file looking like this …
CREATE TABLE `ezxsubscription` (
`id` int(11) NOT NULL auto_increment,
`version_status` int(11) NOT NULL default '0',
`subscriptionlist_id` int(11) default '0',
`email` varchar(255) default '',
`hash` varchar(255) default '',
`status` int(11) default '0',
`vip` int(11) default '0',
`last_active` int(11) NOT NULL default '0',
`output_format` varchar(255) default '',
`creator_id` int(11) NOT NULL default '0',
`created` int(11) NOT NULL default '0',
`confirmed` int(11) NOT NULL default '0',
`approved` int(11) NOT NULL default '0',
`removed` int(11) NOT NULL default '0',
`user_id` int(11) default '0',
`bounce_count` int(11) default '0',
PRIMARY KEY (`id`,`version_status`)
) TYPE=MyISAM;
--
-- Dumping data for table `ezxsubscription`
--
INSERT INTO `ezxsubscription` VALUES (454,1,5,'name#surname.com','31b6bde64e1282ba82d7d0c8ad6ebaa9',2,0,0,'2',1127,1142840848,0,1142840868,0,2633,0);
INSERT INTO `ezxsubscription` VALUES (1,1,5,'name.name#gmail.com','fff4b8d75ecdfad43ed8c89444939cfb',2,0,0,'2',10,1141956489,1141956489,1141956489,0,14,0);
…
The database has like 400 users in it that I couldn't sort by hand. I need to have the database file to be in the format of the predefined CSV above. Where the email-address should be the user_login and user_name. The rest of the old database like approved, removed or whatever can simply be disregarded. It's just about the user-login and the passwords.
Any clever idea on how to do so? The aim is to import all the old users of the old database to the new database (based on Wordpress).
I would really appreciate some tips or tricks.
Thank you in advance.
Matt
Update:
This is the old format of the ezxsubscription table.
"id","version_status","subscriptionlist_id","email","hash","status","vip","last_active","output_format","creator_id","created","confirmed","approved","removed","user_id","bounce_count"
1,1,5,"andreas#gmail.at","fff4b8d75ecdfad43ed8c89444939cfb",2,0,0,2,10,1141956489,1141956489,1141956489,0,14,0
All I wanna do now is convert this users to the new wordpress format:
"ID","user_login","user_pass","user_nicename","user_email","user_url","user_registered","user_activation_key","user_status","display_name"
1,"local","$P$BCf7QHr3rsp0YNZd7eMfYgncRZmR6j0","local","temp#local.dev","","2013-03-14 18:46:29","",0,"local"
I do even have problems understanding what the hash in the old db means.
All I wanna do is copy all existing old users to the new wordpress database so that I have the e-mail addresses and the passwords, so that they are able to login again.
So I want to make the old email the username of the new wordpress db. And the hash should probably be the new user_pass in wordpress. However I only hope that the hash is the password of the current login — I don't even know for sure.
Kind Regards,
Matt

Use SELECT ... INTO OUTFILE syntax to create a CSV file from your old table with a query like this
SELECT *
INTO OUTFILE '/path/to/your/file.csv'
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\n'
FROM
(
SELECT 'id', 'user_login', 'user_pass', 'user_nicename','user_email','user_url','user_registered','user_activation_key','user_status','display_name'
UNION ALL
SELECT id, email, hash, SUBSTRING_INDEX(email, '#', 1), email, '', created, '', status, SUBSTRING_INDEX(email, '#', 1)
FROM ezxsubscription
) q
Content of the created file:
"id","user_login","user_pass","user_nicename","user_email","user_url","user_registered","user_activation_key","user_status","display_name"
"454","name#surname.com","31b6bde64e1282ba82d7d0c8ad6ebaa9","name","name#surname.com","","1142840848","","2","name"
"1","name.name#gmail.com","fff4b8d75ecdfad43ed8c89444939cfb","name.name","name.name#gmail.com","","1141956489","","2","name.name"

Related

WordPress installation is triggered twice after computer restart

I'm working on localhost doing some pages with wordpress. Yesterday I've finished to work with some pages, and then I turn off my pc. Today when I turned it on and I've tried to reach some pages (after have turned theApache and Mysql services on) wordpress say i need to install the framework again.
This is the 1st error i get:
Well I just put the same name etc etc... and the next error is this :
WordPress database error: [Table 'wp_users' already exists]
CREATE TABLE wp_users ( ID bigint(20) unsigned NOT NULL auto_increment, user_login varchar(60) NOT NULL default '', user_pass varchar(255) NOT NULL default '', user_nicename varchar(50) NOT NULL default '', user_email varchar(100) NOT NULL default '', user_url varchar(100) NOT NULL default '', user_registered datetime NOT NULL default '0000-00-00 00:00:00', user_activation_key varchar(255) NOT NULL default '', user_status int(11) NOT NULL default '0', display_name varchar(250) NOT NULL default '', PRIMARY KEY (ID), KEY user_login_key (user_login), KEY user_nicename (user_nicename), KEY user_email (user_email) ) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci
WordPress database error: [Table 'wp_usermeta' already exists]
CREATE TABLE wp_usermeta ( umeta_id bigint(20) unsigned NOT NULL auto_increment, user_id bigint(20) unsigned NOT NULL default '0', meta_key varchar(255) default NULL, meta_value longtext, PRIMARY KEY (umeta_id), KEY user_id (user_id), KEY meta_key (meta_key(191)) ) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci
I tried everything and I can't solve the problem. what can I do? I don´t want to start from scratch with the webpages.
If the database already exists, most likely you have not edited the wp-config.php yet (using any text editor), where you have to insert your database data, like server, db name, db password, db user.
Here are some more details about this: https://wordpress.org/support/article/how-to-install-wordpress/#step-3-set-up-wp-config-php
Seems you are trying to CREATE tables in the DB which already exist, if you need the users already present on db, my suggestion is to manually backup the table (by dumping its), remove its (after the dump), and then re-insert them.
You can easly do that (directly on xampp) by using the PHPMYADMIN panel:
Open XAMPP Control Panel
Start Apache and Mysql Services and go to http://localhost/phpmyadmin/
Here you should be able to find the WordPress Database (in the menù at the left), open it and look for the table citizen by the logs.
export all of them (you should find the export button in the navigation bar upside)
Delete the table after exported.
Now the CREATE command should works fine. You can re-import the table exported at the point 3 after this procedure

No Able to show content of website on joomla

I have pushed my joomla website content to bigrock serverto host the website. But i am getting the error while hitting the url of website.
Below is the error:
Error displaying the error page: Application Instantiation Error: Table 'resoninr_jooml29.resoninr_session' doesn't exist SQL=SELECT session_id FROM resoninr_session WHERE session_id = 'gqcktvtop2kv33lc28ikjrhi44' LIMIT 0, 1
I am new to joomla and database. Can anybody please help me out to resolve the issue.
Thanks in advance.
if you have all table prefix in the db is "jos83" then replace the value of "$dbprefix" variable with the same prefix as in db("jos83") in the configuration.php at the root of joomla site.
It depends how you have transferred the files and database. Through Akeeba or any third party extension or manually. If you transferred manually then there is less chance of change in public $dbprefix = 'jos83_'; present in configuration file, as you need to change just the database user, database name and hostname in configuration.php file.
Suppose you did through Akeeba then it will ask for a new prefix during site restoration. Suppose you did through Akeeba and still it doesnt work then your session table may be damaged. And if session table is damaged Joomla wont load. You have to manully delete the table and recreate it in phpmyadmin using this sql command
DROP TABLE IF EXISTS `resoninr_session`;
CREATE TABLE IF NOT EXISTS `resoninr_session` (
`username` varchar(150) default '',
`time` varchar(14) default '',
`session_id` varchar(200) NOT NULL default '0',
`guest` tinyint(4) default '1',
`userid` int(11) default '0',
`usertype` varchar(50) default '',
`gid` tinyint(3) unsigned NOT NULL default '0',
`client_id` tinyint(3) unsigned NOT NULL default '0',
`data` longtext,
PRIMARY KEY (`session_id`(64)),
KEY `whosonline` (`guest`,`usertype`),
KEY `userid` (`userid`),
KEY `time` (`time`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
Some helpful links
http://forum.joomla.org/viewtopic.php?t=362525
why does joomla 2.5 session table corrupt?
https://www.ostraining.com/blog/joomla/joomla-session-crashed/

Why does this fail on some MySQL installs and not on other MySQL/MariaDB installs

FINAL: I'm closing this question because there is no way for me to reproduce the condition and trying to find a similar occurrence has proven fruitless.
A work-around using perl is being implemented to circumvent the failure that only some mysql installs are experiencing.
UPDATE: I've had another case and verified that this query is returning 'OFF'
SELECT `VARIABLE_VALUE` FROM `information_schema`.`GLOBAL_VARIABLES` WHERE `VARIABLE_NAME` LIKE 'INNODB_STRICT_MODE'
I'm having a hard time tracking down a query issue...
This is part of an update script that I made to install new tables and import data from old ones - if they exist. This is from a procedure that is installed, and then called, by the update script:
In this snippet, the table bot_inventories is created, then a check is performed to see if the old schema table exists. If it does, the old 'base' data is then copied to the new table.
The suspect query is the 'inner' if near the bottom:
CREATE TABLE `bot_inventories` (
`inventories_index` INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
`bot_id` INT(11) UNSIGNED NOT NULL DEFAULT '0',
`slot_id` MEDIUMINT(7) UNSIGNED NOT NULL DEFAULT '0',
`item_id` INT(11) UNSIGNED NULL DEFAULT '0',
`inst_charges` TINYINT(3) UNSIGNED DEFAULT 0,
`inst_color` INT(11) UNSIGNED NOT NULL DEFAULT '0',
`inst_no_drop` TINYINT(1) UNSIGNED NOT NULL DEFAULT '0',
`inst_custom_data` TEXT NULL,
`ornament_icon` INT(11) UNSIGNED NOT NULL DEFAULT '0',
`ornament_id_file` INT(11) UNSIGNED NOT NULL DEFAULT '0',
`ornament_hero_model` INT(11) NOT NULL DEFAULT '0',
`augment_1` MEDIUMINT(7) UNSIGNED NOT NULL DEFAULT '0',
`augment_2` MEDIUMINT(7) UNSIGNED NOT NULL DEFAULT '0',
`augment_3` MEDIUMINT(7) UNSIGNED NOT NULL DEFAULT '0',
`augment_4` MEDIUMINT(7) UNSIGNED NOT NULL DEFAULT '0',
`augment_5` MEDIUMINT(7) UNSIGNED NOT NULL DEFAULT '0',
`augment_6` MEDIUMINT(7) UNSIGNED NOT NULL DEFAULT '0',
PRIMARY KEY (`inventories_index`),
KEY `FK_bot_inventories_1` (`bot_id`),
CONSTRAINT `FK_bot_inventories_1` FOREIGN KEY (`bot_id`) REFERENCES `bot_data` (`bot_id`)
) ENGINE=InnoDB;
IF ((SELECT COUNT(*) FROM `information_schema`.`TABLES` WHERE `TABLE_SCHEMA` = DATABASE() AND `TABLE_NAME` = 'botinventory') > 0) THEN
INSERT INTO `bot_inventories` (
`inventories_index`,
`bot_id`,
`slot_id`,
`item_id`,
`inst_charges`,
`inst_color`,
`inst_no_drop`,
`augment_1`,
`augment_2`,
`augment_3`,
`augment_4`,
`augment_5`
)
SELECT
bi.`BotInventoryID`,
bi.`BotID`,
bi.`SlotID`,
bi.`ItemID`,
bi.`charges`,
bi.`color`,
bi.`instnodrop`,
bi.`augslot1`,
bi.`augslot2`,
bi.`augslot3`,
bi.`augslot4`,
bi.`augslot5`
FROM `botinventory` bi
INNER JOIN `bot_data` bd
ON bi.`BotID` = bd.`bot_id`;
IF ((SELECT COUNT(*) FROM `information_schema`.`COLUMNS` WHERE `TABLE_SCHEMA` = DATABASE() AND `TABLE_NAME` = 'botinventory' AND `COLUMN_NAME` = 'augslot6') > 0) THEN
UPDATE `bot_inventories` bi
INNER JOIN `botinventory` bio
ON bi.`inventories_index` = bio.`BotInventoryID`
SET bi.`augment_6` = bio.`augslot6`
WHERE bi.`bot_id` = bio.`BotID`;
END IF;
RENAME TABLE `botinventory` TO `botinventory_old`;
END IF;
I have run this script myself dozens of times on both 'clean' and convertible (existing and populated botinventory table) databases with no problems.
The script executes properly on both MySQL v5.1.73 and MariaDB v10.0.22 (both win x86)
However, I've had two reported cases of failure surrounding the 'inner' if statement.
That check is in place to catch any 'customization' that may have been done by an admin and import their data.
The failed cases are reporting: "ERROR 1136 (21S01): Column count doesn't match value count at row 1"
I walked one of the admins through the manual installation of this update and found the problem to be with the 'if' query.
EDIT: It's acting like the query inside of the con check is firing off - defeating the purpose of the check itself (NULL is greater than '0'?)
(The walk-through admin is running MySQL v5.1.x as x86 on windows server 2012..the other is running MySQL v5.1.x as win (7?) x86.)
It seems to report that error when 'augslot6' does not exist - opposite behavior of what the check is in place to prevent.
I'm just not sure whether this is a syntax/methodology issue or an actual database code bug.
Searching for similar cases almost exclusively turns up results for column-entry count mis-matches, not why an if statement proves true with a false/null result...
Any suggestions?
EDIT: I am aware of what ANSI SQL code '21S01' indicates and how it could be interpreted to mean the missing augslot6 column. But, if so, why is the 'if' query proving 'true' when it should be 'false' on some systems?
EDIT: I wasn't able to perform the failing test myself as I don't have access to these systems.

import csv file with data to mysql

I'm trying to import csv file to MYSQL, and I have the following schema.
CREATE TABLE `monitor` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`time` time DEFAULT NULL,
`domain_name` text,
`cpu_ns` int(11) DEFAULT NULL,
`cpu_percentage` int(11) DEFAULT NULL,
`mem_bytes` int(11) DEFAULT NULL,
`mem_percentage` int(11) DEFAULT NULL,
`block_rdby` int(11) DEFAULT NULL,
`block_wrby` int(11) DEFAULT NULL,
`net_rxby` int(11) DEFAULT NULL,
`net_wrby` int(11) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
I'm having issues importing a file with data presented as follows.
16:48:27,2,s-1-VM,220000000.,0.448204684384,262144,0,0,0,60,0
16:48:30,2,s-1-VM,260000000.,0.528932926209,262144,0,0,16384,300,0
16:48:33,2,s-1-VM,300000000.,0.609786677944,262144,0,0,0,180,0
16:48:37,2,s-1-VM,290000000.,0.59000206364,262144,0,0,16384,120,0
16:48:40,2,s-1-VM,270000000.,0.54985661784,262144,0,0,0,649,425
16:48:43,2,s-1-VM,310000000.,0.631207212346,262144,0,0,0,180,0
16:48:46,2,s-1-VM,220000000.,0.44728232907,262144,0,0,20480,60,0
16:48:49,2,s-1-VM,200000000.,0.407008216196,262144,0,0,0,300,0
16:48:52,2,s-1-VM,250000000.,0.508946559213,262144,0,0,0,240,0
16:48:55,2,s-1-VM,240000000.,0.488674160215,262144,0,0,0,120,0
How can import this to my database?
I have tried the following and I get lots of warnings.
LOAD DATA LOCAL INFILE '/tmp/domain2.csv' INTO TABLE vtop FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';
your help is highly appreciated.
Thank you
If I understand http://dev.mysql.com/doc/refman/5.1/de/load-data.html correctly, you should set ID to null (that makes it auto_increment) via
"SET id=NULL "
at the end of the statement. Otherwise column counts and column orders have to match perfectly.
But your columns don't match at all (what is the "2" at position 2?). So create a temp-Table with the structure of your CSV and then assign via insert into ... select the matching columns.
MySQL isn't an AI and can't figure out that 16:48:27 should go into the the time field - it'll be trying to stuff that into id instead.
You need to explictly map the columns in your CSV file to the fields they should go into in the table:
LOAD DATA .... (time, domain_name, x, y, z, foo, bar)

What's is better for a members database?

Hi and thanx for reading my post i am having a little trouble learning my database in mysql.
Now i have it set up already but recently, but i had another person tell me my members table is slow and useless if i intend to have a lots of members!
I have looked it over a lot of times and did some google searches but i don't see anything wrong with it, maybe because i am new at it? can one of you sql experts look it over and tell me whats wrong with it please :)
--
-- Table structure for table `members`
--
CREATE TABLE IF NOT EXISTS `members` (
`userid` int(9) unsigned NOT NULL AUTO_INCREMENT,
`username` varchar(20) NOT NULL DEFAULT '',
`password` longtext,
`email` varchar(80) NOT NULL DEFAULT '',
`gender` int(1) NOT NULL DEFAULT '0',
`ipaddress` varchar(80) NOT NULL DEFAULT '',
`joinedon` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`acctype` int(1) NOT NULL DEFAULT '0',
`acclevel` int(1) NOT NULL DEFAULT '0',
`birthdate` date DEFAULT NULL,
`warnings` int(1) NOT NULL DEFAULT '0',
`banned` int(1) NOT NULL DEFAULT '0',
`enabled` int(1) NOT NULL DEFAULT '0',
`online` int(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`userid`),
UNIQUE KEY `username` (`username`),
UNIQUE KEY `emailadd` (`emailadd`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=19 ;
--
-- Dumping data for table `members`
--
It's going to be a site for faqs/tips for games, i do expect to get lots of members at one point later on but i thought i would ask to make sure it's all ok, thanx again peace.
Did the other person explain why they think it is slow and useless?
Here's a few things that I think could be improved:
email should be longer - off the top of my head, 320 should be long enough for most email addresses, but you might want to look that up.
If the int(1) fields are simple on/off fields, then they could be tinyint(1) or bool instead.
As #cularis points out, the ipaddress field might not be the appropriate type. INT UNSIGNED is better than varchar for IPv4. You can use INET_ATON() and INET_NTOA() for conversion. See:
Best Field Type for IP address?
How to store IPv6-compatible address in a relational database
As #Delan Azabani points out, your password field is too long for the value you are storing. MD5 produces a 32 character string, so varchar(32) will be sufficient. You could switch to the more secure SHA2, and use the MySQL 'SHA2()' function.
Look into using the InnoDB database engine instead of MyISAM. It offers foreign key constraints, row-level locking and transactions, amongst other things. See Should you move from MyISAM to Innodb ?.
I don't think it's necessarily slow, but I did notice that among all other text fields where you used varchar, you used longtext for the password field. This seems like you are going to store the password in the database -- don't do this!
Always take a fixed-length cryptographic hash (using, for example, SHA-1 or SHA-2) of the user's password, and put that into the database. That way, if your database server is compromised, the users' passwords are not exposed.
Apart from what #Delan said, I noted that;
JoinedOn column defined as ON UPDATE CURRENT_TIMESTAMP. If you need to maintain only the date joined, you should not update the field when the records been updated.
IPAddress column is VARCHAR(80). If you store IPv4 type IP addresses, this will be too lengthy.
Empty string ('') as DEFAULT for NOT NULL columns. Not good if intention is to have a value (other than '') on the field.
Empty string ('') as DEFAULT for UNIQUE Fields. This contradicts the contraints enforced if your intention is to have a Unique Value (other than '').