Deleting All Non-Recently Updated Keys/Inserts MySQL - mysql

I'm trying to create a DELETE statement in Java where after updating and inserting all new items into the DB, it deletes any old ones that were not affected by the update.
Reasoning: I'm hosting a game where items are saved upon logging out. Items are either updated or inserted depending on whether or not their reference exists. However, it is possible that a player could have dropped an item inside the game, thus no longer having ownership of it. I need to delete the reference to this item now that the player no longer has it.
tl;dr: Trying to write a DELETE FROM WHERE NOT IN for all non-updated objects belonging to the character
Table definition:
CREATE TABLE `inventoryitems` (
`inventoryitemid` int(10) unsigned NOT NULL AUTO_INCREMENT,
`type` tinyint(3) unsigned NOT NULL,
`characterid` int(11) DEFAULT NULL,
`accountid` int(11) DEFAULT NULL,
`itemid` int(11) NOT NULL DEFAULT '0',
`inventorytype` int(11) NOT NULL DEFAULT '0',
`position` int(11) NOT NULL DEFAULT '0',
`quantity` int(11) NOT NULL DEFAULT '0',
`owner` tinytext NOT NULL,
`petid` int(11) NOT NULL DEFAULT '-1',
`flag` int(11) NOT NULL,
`expiration` bigint(20) NOT NULL DEFAULT '-1',
`giftFrom` varchar(26) NOT NULL,
PRIMARY KEY (`inventoryitemid`),
KEY `CHARID` (`characterid`),
KEY `inventorytype` (`inventorytype`),
KEY `type` (`type`),
KEY `accountid` (`accountid`)
) ENGINE=InnoDB AUTO_INCREMENT=79597880 DEFAULT CHARSET=latin1

Related

Extremely slow query- Using google sql cloud

Is there a way I can speed this up? Right now it's taking an unbelievably insane amount of time to query.
SELECT trades.*, trader1.user_name as trader1_name,
trader2.user_name as trader2_name FROM trades
LEFT JOIN logs_players trader1 ON trader1.user_id = trader1_account_id
LEFT JOIN logs_players trader2 ON trader2.user_id = trader2_account_id
ORDER BY time_added
LIMIT 20 OFFSET 0;
I've done as much as I could in terms of searching online for a solution. Or even just trying to get some more information why it's taking so long to execute.
The query takes about 45 seconds or so to complete.
Create statements:
CREATE TABLE `trades` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`trader1_account_id` int(11) DEFAULT NULL,
`trader2_account_id` int(11) DEFAULT NULL,
`trader1_value` bigint(20) DEFAULT NULL,
`trader2_value` bigint(20) DEFAULT NULL,
`trader1_ip` varchar(16) DEFAULT NULL,
`trader2_ip` varchar(16) DEFAULT NULL,
`world` int(11) DEFAULT NULL,
`x` int(11) DEFAULT NULL,
`z` int(11) DEFAULT NULL,
`level` int(11) DEFAULT NULL,
`trader1_user` varchar(12) DEFAULT NULL,
`trader2_user` varchar(12) DEFAULT NULL,
`time_added` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=0 DEFAULT CHARSET=utf8
CREATE TABLE `logs_players` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) DEFAULT NULL,
`user_name` varchar(20) DEFAULT NULL,
`world_stage` varchar(20) DEFAULT NULL,
`world_type` varchar(20) DEFAULT NULL,
`bank` longtext,
`inventory` longtext,
`equipment` longtext,
`total_wealth` mediumtext,
`total_play_time` mediumtext,
`rights` int(11) DEFAULT NULL,
`icon` int(11) DEFAULT NULL,
`ironmode` int(11) DEFAULT NULL,
`x` int(11) DEFAULT NULL,
`z` int(11) DEFAULT NULL,
`level` int(11) DEFAULT NULL,
`last_ip` varchar(16) DEFAULT NULL,
`last_online` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`muted_until` timestamp NULL DEFAULT NULL,
`banned_until` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=0 DEFAULT CHARSET=utf8
I filled a sample database with 10k rows each, and found that a few indexes were what you needed:
ALTER TABLE `logs_players` ADD INDEX(`user_id`);
ALTER TABLE `trades` ADD INDEX(`time_added`);
The main index we need is an index on user_id. Changing that we went from a query time of 20.1390 seconds, to 0.0130 seconds:
We can even get that down further, by adding an index on time_added to make sorting a lot faster, now we ended up with an impressive query time:
Do some research on indexes! A simple EXPLAIN query would show you that you're using filesort (Which is rather bad!):
After indexes, this looks a lot better:

mysqldiff tool drops indexes and adds duplicates and causes errors

I'm using the mysqldiff tool (v1.5.3) from MySQL to synchronize the schema between two databases. The databases should have the same schema, but would like to produce a file that only has the ALTER TABLE differences between them so it can (after thorough testing) be run in production.
The problem is that the mysqldiff tool produces some weirdness. For example:
ALTER TABLE `database1`.`ACCOUNT_TRANSACTIONS`
DROP INDEX MATCHING_TRANSACTION_ID,
DROP INDEX PAYEE_ID,
ADD INDEX CHECK_NUMBER (ACCOUNT_ID),
ADD INDEX PAYEE_ID (PAYEE_ID),
ADD INDEX CHECK_NUMBER (PAYEE_ID,CHECK_NUMBER);
It seems to not only drop the indexes, but tries to recreate the new index through two statements that produce an error. The final schema should have CHECK_NUMBER (ACCOUNT_ID, PAYEE_ID, CHECK_NUMBER) as the index.
What might be causing this?
Final version:
CREATE TABLE `ACCOUNT_TRANSACTIONS` (
`TRANSACTION_ID` int(12) NOT NULL AUTO_INCREMENT,
`ACCOUNT_ID` int(1) DEFAULT NULL,
`REF_NUMBER` varchar(15) DEFAULT NULL,
`PAYEE_ID` int(6) DEFAULT NULL,
`ACCOUNT_CHART_ID` int(12) DEFAULT NULL,
`TRANSACTION_AMOUNT` decimal(10,2) DEFAULT NULL,
`TRANSACTION_DATE` date DEFAULT NULL,
`MEMO` varchar(100) DEFAULT NULL,
`ACCOUNT_BALANCE` decimal(10,2) DEFAULT NULL,
`DEBIT_OR_CREDIT` char(1) DEFAULT 'D',
`MATCHING_TRANSACTION_ID` int(11) DEFAULT '0',
`RECONCILED_DATE` date DEFAULT NULL,
`RECONCILED_EMPLOYEE_ID` int(12) DEFAULT NULL,
`TIME_UPDATED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`CHECK_NUMBER` bigint(20) unsigned DEFAULT NULL,
`STATEMENT_ID` int(12) DEFAULT NULL,
PRIMARY KEY (`TRANSACTION_ID`),
KEY `PAYEE_ID` (`PAYEE_ID`),
KEY `CHECK_NUMBER` (`CHECK_NUMBER`,`ACCOUNT_ID`,`PAYEE_ID`)
) ENGINE=InnoDB AUTO_INCREMENT=xxx DEFAULT CHARSET=latin1
Original version (needs to be altered to be final version)
CREATE TABLE `ACCOUNT_TRANSACTIONS` (
`TRANSACTION_ID` int(12) NOT NULL AUTO_INCREMENT,
`ACCOUNT_ID` int(1) DEFAULT NULL,
`REF_NUMBER` varchar(15) DEFAULT NULL,
`PAYEE_ID` int(6) DEFAULT NULL,
`ACCOUNT_CHART_ID` int(12) DEFAULT NULL,
`TRANSACTION_AMOUNT` decimal(10,2) DEFAULT NULL,
`TRANSACTION_DATE` date DEFAULT NULL,
`MEMO` varchar(100) DEFAULT NULL,
`ACCOUNT_BALANCE` decimal(10,2) DEFAULT NULL,
`DEBIT_OR_CREDIT` char(1) DEFAULT 'D',
`MATCHING_TRANSACTION_ID` int(11) DEFAULT '0',
`RECONCILED_DATE` date DEFAULT NULL,
`RECONCILED_EMPLOYEE_ID` int(12) DEFAULT NULL,
`TIME_UPDATED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`CHECK_NUMBER` bigint(20) unsigned DEFAULT NULL,
`STATEMENT_ID` int(12) DEFAULT NULL,
PRIMARY KEY (`TRANSACTION_ID`),
KEY `PAYEE_ID` (`PAYEE_ID`),
KEY `MATCHING_TRANSACTION_ID` (`MATCHING_TRANSACTION_ID`)
) ENGINE=MyISAM AUTO_INCREMENT=xxx DEFAULT CHARSET=latin1

Hibernate - How to map two tables on two columns?

I have two tables:
CREATE TABLE `framework_files` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`fileName` varchar(777) DEFAULT NULL,
`fileSize` float DEFAULT NULL,
`filePath` varchar(777) DEFAULT NULL,
`fileUrl` varchar(777) DEFAULT NULL,
`linkedObjectID` int(11) DEFAULT NULL,
`linkedObjectName` varchar(250) DEFAULT NULL,
`active` int(11) DEFAULT '0',
`uploadDate` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`uploadersIP` varchar(77) DEFAULT NULL,
`uploadersIPXFORWARDEDFOR` varchar(77) DEFAULT NULL,
`userName` varchar(777) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `q_idx` (`linkedObjectID`,`linkedObjectName`)
) ENGINE=InnoDB AUTO_INCREMENT=110 DEFAULT CHARSET=utf8;
And second table
CREATE TABLE `publicSitePortfolioWorks` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`mainName` varchar(100) DEFAULT NULL,
`tableName` varchar(250) DEFAULT 'publicSitePortfolioWorks',
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=9 DEFAULT CHARSET=utf8;
I need to map publicSitePortfolioWorks to framework_files on framework_files.linkedObjectID=publicSitePortfolioWorks.id and framework_files.linkedObjectName=publicSitePortfolioWork.tableName .
I.e. publicSitePortfolioWorks have many objects of framework_files. All child objects must be in Set, like
private Set<FrameworkFiles> fileTEST = new HashSet<>(0);
How to map this Set in hbm.xml file? At this moment no foreign key between these two tables.

Migrate dataset from two MySQL tables where auto increment id key is the relation

I have two MySQL tables in my development database. (Some might recognize Drupal here..)
CREATE TABLE `node` (
`nid` int(10) unsigned NOT NULL AUTO_INCREMENT,
`vid` int(10) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`nid`),
UNIQUE KEY `vid` (`vid`),
);
CREATE TABLE `content_type_fenykep` (
`vid` int(10) unsigned NOT NULL DEFAULT '0',
`nid` int(10) unsigned NOT NULL DEFAULT '0',
`field_fenykep_fid` int(11) DEFAULT NULL,
`field_galeria_nid` int(10) unsigned DEFAULT NULL,
PRIMARY KEY (`vid`), KEY `nid` (`nid`),
KEY `field_galeria_nid` (`field_galeria_nid`)
);
CREATE TABLE `files` (
`fid` int(10) unsigned NOT NULL AUTO_INCREMENT,
`uid` int(10) unsigned NOT NULL DEFAULT '0',
`filename` varchar(255) NOT NULL DEFAULT '',
`filepath` varchar(255) NOT NULL DEFAULT '',
PRIMARY KEY (`fid`),  KEY `uid` (`uid`),
);
They are connected by the content_type_fenykep.field_fenykep_fid field that refers to files.fid.
I would like to migrate a set of records from content_type_fenykep with the appropriate files records as well to a different, let's say production database. My problem is that I need to change the fids in every content_type_fenykep record because both tables have their different autoincrement states.
My first idea would be writing a program for this problem, but I am wondering if this can be solved only with MySQL or not.
Do you have any ideas how this could be achieved?

InnoDB foreign key definition cannot resolve column name

I have a problem creating a foreign key constraint between two tables.
Here is the first table:
CREATE TABLE `agews_rifiuti_cer` (
`id_cer` int(10) unsigned NOT NULL AUTO_INCREMENT,
`agews_id` int(10) unsigned DEFAULT '0',
`livello` tinyint(4) DEFAULT '1',
`codice` varchar(10) DEFAULT NULL,
`descrizione` varchar(255) DEFAULT NULL,
`note` text,
`flag_pericoloso` tinyint(1) DEFAULT '0',
`id_cliente` int(10) unsigned DEFAULT '1',
`flag_modificato` char(1) DEFAULT 'N',
PRIMARY KEY (`id_cer`),
KEY `fk_id_cliente_agews_sgs_codici_cer` (`id_cliente`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
And here is the second one:
CREATE TABLE `lin_98_47_rifiuti` (
`id_rifiuto` int(10) unsigned NOT NULL AUTO_INCREMENT,
`id_azienda` int(10) unsigned NOT NULL DEFAULT '1',
`id_sede` int(10) unsigned NOT NULL DEFAULT '1',
`revisione_documento` int(10) unsigned NOT NULL DEFAULT '0',
`rifiuto` varchar(255) DEFAULT NULL,
`codice_cer` varchar(10) DEFAULT NULL,
`nome_interno` varchar(255) DEFAULT NULL,
`descrizione` text,
`materie_prime` text,
`contenitore` text,
`deposito` text,
`data_ultima_analisi` date DEFAULT NULL,
`stato_fisico` enum('Solido pulverulento','Solido non pulverulento','Fangoso palabile','Liquido') DEFAULT 'Solido non pulverulento',
`quantita` float(9,1) DEFAULT '0.0',
`unita_misura` enum('Kg','l','mc') DEFAULT 'Kg',
`id_pericolo` int(10) unsigned DEFAULT NULL,
`destino` enum('Recupero','Smaltimento') DEFAULT NULL,
`id_recupero` int(10) unsigned DEFAULT NULL,
`id_smaltimento` int(10) unsigned DEFAULT NULL,
`id_cer` int(10) unsigned DEFAULT NULL,
`immagine` varchar(255) DEFAULT NULL,
`image_type` varchar(20) DEFAULT NULL,
`image_content` mediumblob,
`image_size_x` smallint(5) unsigned DEFAULT '0',
`image_size_y` smallint(5) unsigned DEFAULT '0',
`flag_storico` tinyint(1) DEFAULT '0',
`id_responsabile` int(10) unsigned DEFAULT '0',
`nome_responsabile` varchar(255) DEFAULT '0',
`id_ultima_modifica` int(10) unsigned DEFAULT '0',
`create_log` tinyint(1) DEFAULT '1',
PRIMARY KEY (`id_rifiuto`,`id_azienda`,`id_sede`,`revisione_documento`),
KEY `fk_main_lin_98_47_rifiuti` (`id_azienda`,`id_sede`,`revisione_documento`),
KEY `fk_id_responsabile_lin_98_47_rifiuti` (`id_responsabile`,`id_azienda`,`id_sede`,`revisione_documento`,`nome_responsabile`),
KEY `fk_id_pericolo_lin_98_47_rifiuti` (`id_pericolo`),
KEY `fk_id_recupero_lin_98_47_rifiuti` (`id_recupero`),
KEY `fk_id_smaltimento_lin_98_47_rifiuti` (`id_smaltimento`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
The problem happens the I try to do this:
ALTER TABLE lin_98_47_rifiuti ADD CONSTRAINT fk_id_cer_lin_98_47_rifiuti FOREIGN KEY (id_cer) REFERENCES agews_rifiuti_cer(id_cer) ON UPDATE CASCADE ON DELETE CASCADE;
And I get this error:
#1005 - Can't create table 'db_626suite.#sql-71c_13d5' (errno: 150)
While the command SHOW INNODB STATUS says:
Error in foreign key constraint of table db_626suite/#sql-71c_13d5:
FOREIGN KEY (id_cer) REFERENCES agews_rifiuti_cer(id_cer) ON UPDATE CASCADE ON DELETE CASCADE:
Cannot resolve column name close to:
) ON UPDATE CASCADE ON DELETE CASCADE
But the syntax and fields definitions seem correct to me. What am I doing wrong?
EDIT:
Marc B suggested that it could be due to the fact that id_cer is defined as NOT NULL in agews_rifiuti_cer, but I do not think this is the case, in fact please consider this other table:
CREATE TABLE `agews_rifiuti_pericolo` (
`id_pericolo` int(10) unsigned NOT NULL AUTO_INCREMENT,
`agews_id` int(10) unsigned DEFAULT '0',
`codice` varchar(255) DEFAULT NULL,
`pericolo` varchar(255) DEFAULT NULL,
`note` text,
`id_cliente` int(10) unsigned DEFAULT '1',
`flag_modificato` char(1) DEFAULT 'N',
PRIMARY KEY (`id_pericolo`),
KEY `codice_rifiuti_recupero` (`codice`),
KEY `fk_id_cliente_rifiuti_pericolo` (`id_cliente`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Here id_pericolo is defined as NOT NULL as id_cer above, and yet this works perfectly:
ALTER TABLE lin_98_47_rifiuti ADD CONSTRAINT fk_id_pericolo_lin_98_47_rifiuti FOREIGN KEY (id_pericolo) REFERENCES agews_rifiuti_pericolo(id_pericolo) ON UPDATE CASCADE ON DELETE SET NULL;
even if in lin_98_47_rifiuti the field id_pericolo is defined as DEFAULT NULL
EDIT 2:
I just tried this minimal setup:
CREATE TABLE `cer` (
`id_cer` int(10) unsigned NOT NULL AUTO_INCREMENT,
`codice` varchar(10) DEFAULT NULL,
PRIMARY KEY (`id_cer`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `rifiuti` (
`id_rifiuto` int(10) unsigned NOT NULL AUTO_INCREMENT,
`id_cer` int(10) unsigned NULL DEFAULT NULL,
`rifiuto` varchar(10) DEFAULT NULL,
PRIMARY KEY (`id_rifiuto`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
ALTER TABLE rifiuti ADD FOREIGN KEY (id_cer) REFERENCES cer(id_cer) ON UPDATE CASCADE ON DELETE CASCADE;
and it all worked correctly, yet I do not understand what the problem might be in my original query.
Also I think this proves that is not a matter of defining the fields as NOT NULL or DEFAULT NULL.
You've defined id_cer as default null in the lin_98_47 table, but the matching FK field in the other table is defined as not null.
The field definitions have to match EXACTLY for a foreign key relationship to be established, and that includes nullability:
`id_cer` int(10) unsigned NOT NULL AUTO_INCREMENT,
`id_cer` int(10) unsigned DEFAULT NULL,
I really do not understand why, but renaming id_cer to id_codice in all tables, while leaving its definition unaltered, has solved the problem.
So it would seem that MySQL had some kind of problem with that specific field name.