The following throws slow query log.
APN::Notification.
select('apn_notifications.*, devices.device_uid').
joins('INNER JOIN apn_devices ON (apn_notifications.device_id = apn_devices.id) INNER JOIN devices ON (apn_devices.device_id = devices.id)').
where(['apn_notifications.sent_at IS NULL AND apn_notifications.badge > 0 AND devices.customer_id = ? AND devices.device_type IN (?)', customer.id, Object::Device.platform_device_types('ios')])
The output of EXPLAIN
EXPLAIN for: SELECT apn_notifications.*, devices.device_uid FROM `apn_notifications` INNER JOIN apn_devices ON (apn_notifications.device_id = apn_devices.id) INNER JOIN devices ON (apn_devices.device_id = devices.id) WHERE (apn_notifications.disabled_at IS NULL) AND (apn_notifications.sent_at IS NULL AND apn_notifications.badge > 0 AND devices.customer_id = 1 AND devices.device_type IN ('iphone4','ipad','iphone3'))
The Output of 'show create table apn_notifications'
| apn_notifications | CREATE TABLE `apn_notifications` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`device_id` int(11) DEFAULT NULL,
`errors_nb` int(11) DEFAULT NULL,
`device_language` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`sound` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`alert` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`badge` int(11) DEFAULT NULL,
`custom_properties` text COLLATE utf8_unicode_ci,
`sent_at` datetime DEFAULT NULL,
`disabled_at` datetime DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_apn_notifications_on_device_id` (`device_id`)
) ENGINE=InnoDB AUTO_INCREMENT=12984412 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci |
the apn_notifications table has 1.5 million records. So if I try to add index, it is taking much longer time. What is the best way to remove this from slow query?
Also From Mysql 5.6, adding index will not result in any downtime. Am I right?
Often "composite" indexes are better:
apn_notifications: INDEX(device_id, sent_at, badge)
apn_notifications: INDEX(sent_at, badge)
devices: INDEX(customer_id, device_type)
apn_devices is a many-to-many mapping? If so, check that it is following the guidelines here .
I added the following indexes and that reduced lots of time.
add_index :apn_notifications, :sent_at
add_index :apn_notifications, :badge
NOTE: Already indexes were added for foreign keys
Related
I have a very long running MySql query. The query simply joins two tables which are very huge
bizevents - Nearly 34 Million rows
bizevents_actions - Nearly 17 million rows
Here is the query:
select
bizevent0_.id as id1_37_,
bizevent0_.json as json2_37_,
bizevent0_.account_id as account_3_37_,
bizevent0_.createdBy as createdB4_37_,
bizevent0_.createdOn as createdO5_37_,
bizevent0_.description as descript6_37_,
bizevent0_.iconCss as iconCss7_37_,
bizevent0_.modifiedBy as modified8_37_,
bizevent0_.modifiedOn as modified9_37_,
bizevent0_.name as name10_37_,
bizevent0_.version as version11_37_,
bizevent0_.fired as fired12_37_,
bizevent0_.preCreateFired as preCrea13_37_,
bizevent0_.entityRefClazz as entityR14_37_,
bizevent0_.entityRefIdAsStr as entityR15_37_,
bizevent0_.entityRefIdType as entityR16_37_,
bizevent0_.entityRefName as entityR17_37_,
bizevent0_.entityRefType as entityR18_37_,
bizevent0_.entityRefVersion as entityR19_37_
from
BizEvent bizevent0_
left outer join BizEvent_actions actions1_ on
bizevent0_.id = actions1_.BizEvent_id
where
bizevent0_.createdOn >= '1969-12-31 19:00:01.0'
and (actions1_.action <> 'SoftLock'
and actions1_.targetRefClazz = 'com.biznuvo.core.orm.domain.org.EmployeeGroup'
and actions1_.targetRefIdAsStr = '1'
or actions1_.action <> 'SoftLock'
and actions1_.objectRefClazz = 'com.biznuvo.core.orm.domain.org.EmployeeGroup'
and actions1_.objectRefIdAsStr = '1')
order by
bizevent0_.createdOn;
Below are the table definitions -- As you see i have defined the indexes well enough on these two tables on all the search columns plus the sort column. But still my queries are running for very very long time. Appreciate any more ideas either with respective indexing.
-- bizevent definition
CREATE TABLE `bizevent` (
`id` bigint(20) NOT NULL,
`json` longtext,
`account_id` int(11) DEFAULT NULL,
`createdBy` varchar(50) NOT NULL,
`createdon` datetime(3) DEFAULT NULL,
`description` varchar(255) DEFAULT NULL,
`iconCss` varchar(50) DEFAULT NULL,
`modifiedBy` varchar(50) NOT NULL,
`modifiedon` datetime(3) DEFAULT NULL,
`name` varchar(255) NOT NULL,
`version` int(11) NOT NULL,
`fired` bit(1) NOT NULL,
`preCreateFired` bit(1) NOT NULL,
`entityRefClazz` varchar(255) DEFAULT NULL,
`entityRefIdAsStr` varchar(255) DEFAULT NULL,
`entityRefIdType` varchar(25) DEFAULT NULL,
`entityRefName` varchar(255) DEFAULT NULL,
`entityRefType` varchar(50) DEFAULT NULL,
`entityRefVersion` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `IDXk9kxuuprilygwfwddr67xt1pw` (`createdon`),
KEY `IDXsf3ufmeg5t9ok7qkypppuey7y` (`entityRefIdAsStr`),
KEY `IDX5bxv4g72wxmjqshb770lvjcto` (`entityRefClazz`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
-- bizevent_actions definition
CREATE TABLE `bizevent_actions` (
`BizEvent_id` bigint(20) NOT NULL,
`action` varchar(255) DEFAULT NULL,
`objectBizType` varchar(255) DEFAULT NULL,
`objectName` varchar(255) DEFAULT NULL,
`objectRefClazz` varchar(255) DEFAULT NULL,
`objectRefIdAsStr` varchar(255) DEFAULT NULL,
`objectRefIdType` int(11) DEFAULT NULL,
`objectRefVersion` int(11) DEFAULT NULL,
`targetBizType` varchar(255) DEFAULT NULL,
`targetName` varchar(255) DEFAULT NULL,
`targetRefClazz` varchar(255) DEFAULT NULL,
`targetRefIdAsStr` varchar(255) DEFAULT NULL,
`targetRefIdType` int(11) DEFAULT NULL,
`targetRefVersion` int(11) DEFAULT NULL,
`embedJson` longtext,
`actions_ORDER` int(11) NOT NULL,
PRIMARY KEY (`BizEvent_id`,`actions_ORDER`),
KEY `IDXa21hhagjogn3lar1bn5obl48gll` (`action`),
KEY `IDX7agsatk8u8qvtj37vhotja0ce77` (`targetRefClazz`),
KEY `IDXa7tktl678kqu3tk8mmkt1mo8lbo` (`targetRefIdAsStr`),
KEY `IDXa22eevu7m820jeb2uekkt42pqeu` (`objectRefClazz`),
KEY `IDXa33ba772tpkl9ig8ptkfhk18ig6` (`objectRefIdAsStr`),
CONSTRAINT `FKr9qjs61id11n48tdn1cdp3wot` FOREIGN KEY (`BizEvent_id`) REFERENCES `bizevent` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;>
By the way we are using Amazon RDS 5.7.33 MySql version. 16 GB RAM and 4 vCPU.
I also did a Explain Extended on the query and below is what it shows. Appreciate any help.
Initially the search of the bizevent_actions didn;t have the indexes defined. I have defined the indexes for them and tried the query but of no use.
One technique that worked for me in a similar situation was abandoning the idea of JOIN completely and switching to queries by PK. More detailed: find out which table in join would give less rows on average if you use only that table and related filter to query; get the primary keys from that table and then query the other one using WHERE pk IN ().
In your case one example would be:
SELECT
bizevent0_.id as id1_37_,
bizevent0_.json as json2_37_,
bizevent0_.account_id as account_3_37_,
...
FROM BizEvent bizevent0_
WHERE
bizevent0_.createdOn >= '1969-12-31 19:00:01.0'
AND bizevent0_.id IN (
SELECT BizEvent_id
FROM BizEvent_actions actions1_
WHERE
actions1_.action <> 'SoftLock'
and actions1_.targetRefClazz = 'com.biznuvo.core.orm.domain.org.EmployeeGroup'
and actions1_.targetRefIdAsStr = '1'
or actions1_.action <> 'SoftLock'
and actions1_.objectRefClazz = 'com.biznuvo.core.orm.domain.org.EmployeeGroup'
and actions1_.objectRefIdAsStr = '1')
ORDER BY
bizevent0_.createdOn;
This assumes that you're not actually willing to select 33+ Mio rows from BizEvent though - your code with LEFT OUTER JOIN would have done exactly this.
I have two databases with identical schemas. The one database is from production, the other is a test database. I'm doing a query against a single table from the database. On the production table the query takes around 4.3 seconds, while on the test database it takes about 130 ms. . However, the production table has less then 50.000 records, while I've seeded the test table with more than 100.000. I've compared the two tables and both have the same indexes. To me, it seems that the problem is in the data. While seeding I tried to generate as random data as possible, so that I can simulate production conditions, but still I couldn't reproduce the slow query.
I looked the the results from EXPLAIN for the two queries. They have significant differences in the last two columns.
Production:
+-------+-------------------------+
| rows | Extra |
+-------+-------------------------+
| 24459 | Using where |
| 46 | Using where; Not exists |
+-------+-------------------------+
Test:
+------+------------------------------------+
| rows | Extra |
+------+------------------------------------+
| 3158 | Using index condition; Using where |
| 20 | Using where; Not exists |
+------+------------------------------------+
The create statement for the table on production is:
CREATE TABLE `usage_logs` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) NOT NULL,
`operation` varchar(30) COLLATE utf8_unicode_ci NOT NULL,
`check_time` datetime NOT NULL,
`check_in_log_id` int(11) DEFAULT NULL,
`daily_usage_id` int(11) DEFAULT NULL,
`duration_units` decimal(11,2) DEFAULT NULL,
`is_deleted` tinyint(1) NOT NULL DEFAULT '0',
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
`facility_id` int(11) NOT NULL,
`notes` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`mac_address` varchar(20) COLLATE utf8_unicode_ci NOT NULL DEFAULT '00:00:00:00:00:00',
`login` varchar(40) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_usage_logs_on_user_id` (`user_id`),
KEY `index_usage_logs_on_check_in_log_id` (`check_in_log_id`),
KEY `index_usage_logs_on_facility_id` (`facility_id`),
KEY `index_usage_logs_on_check_time` (`check_time`),
KEY `index_usage_logs_on_mac_address` (`mac_address`),
KEY `index_usage_logs_on_operation` (`operation`)
) ENGINE=InnoDB AUTO_INCREMENT=145147 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
while the same in the test database is:
CREATE TABLE `usage_logs` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) NOT NULL,
`operation` varchar(30) COLLATE utf8_unicode_ci NOT NULL,
`check_time` datetime NOT NULL,
`check_in_log_id` int(11) DEFAULT NULL,
`daily_usage_id` int(11) DEFAULT NULL,
`duration_units` decimal(11,2) DEFAULT NULL,
`is_deleted` tinyint(1) NOT NULL DEFAULT '0',
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
`facility_id` int(11) NOT NULL,
`notes` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`mac_address` varchar(20) COLLATE utf8_unicode_ci NOT NULL DEFAULT '00:00:00:00:00:00',
`login` varchar(40) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_usage_logs_on_check_in_log_id` (`check_in_log_id`),
KEY `index_usage_logs_on_check_time` (`check_time`),
KEY `index_usage_logs_on_facility_id` (`facility_id`),
KEY `index_usage_logs_on_mac_address` (`mac_address`),
KEY `index_usage_logs_on_operation` (`operation`),
KEY `index_usage_logs_on_user_id` (`user_id`)
) ENGINE=InnoDB AUTO_INCREMENT=104001 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
The full query is:
SELECT `usage_logs`.*
FROM `usage_logs`
LEFT OUTER JOIN usage_logs AS usage_logs_latest ON usage_logs.facility_id = usage_logs_latest.facility_id
AND usage_logs.user_id = usage_logs_latest.user_id
AND usage_logs.mac_address = usage_logs_latest.mac_address
AND usage_logs.check_time < usage_logs_latest.check_time
WHERE `usage_logs`.`facility_id` = 5
AND `usage_logs`.`operation` = 'checkIn'
AND (usage_logs.check_time >= '2018-06-08 00:00:00')
AND (usage_logs.check_time <= '2018-06-08 11:23:05')
AND (usage_logs_latest.id IS NULL)
I execute the query on the same machine against two different databases, so I don't think that other processes are interfering in the result.
What does this result mean and what further steps can I take in order to find out the reason for the big difference in the execution time?
What MySQL version(s) are you using?
There are many factors that lead to the decision by the Optimizer as to
which table to start with; (we can't see if they are different)
which index(es) to use; (we can't see)
etc.
Some of the factors:
the distribution of the index values at the moment,
the MySQL version,
the phase of the moon.
These can also lead to different numbers (estimates) in the EXPLAIN, which may lead to different query plans.
Also other activity in the server can interfere with the availability of CPU/IO/etc. In particular caching of the data can easily show a 10x difference. Did you run each query twice? Is the Query cache turned off? Is innodb_buffer_pool_size the same? Is RAM size the same?
I see Using index condition and no "composite" indexes. Often performance can be improved by providing a suitable composite index. More
I gotta see the query!
Seeding
Random, or not-so-random, rows can influence the Optimizer's choice of which index (etc) to use. This may have led to picking a better way to run the query on 'test'.
We need to see EXPLAIN SELECT ... to discuss this angle further.
Composite indexes
These are likely to help on both servers:
INDEX(facility_id, operation, -- either order
check_time) -- last
INDEX(facility_id, user_id, max_address, check_time, -- any order
id) -- last
There is a quick improvement. Instead of finding all the later rows, but not use the contents of them, use a 'semi-join' which asks of the non-existence of any such rows:
SELECT `usage_logs`.*
FROM `usage_logs`
WHERE `usage_logs`.`facility_id` = 5
AND `usage_logs`.`operation` = 'checkIn'
AND (usage_logs.check_time >= '2018-06-08 00:00:00')
AND (usage_logs.check_time <= '2018-06-08 11:23:05')
AND NOT EXISTS ( SELECT 1 FROM usage_logs AS latest
WHERE usage_logs.facility_id = latest.facility_id
AND usage_logs.user_id = latest.user_id
AND usage_logs.mac_address = latest.mac_address
AND usage_logs.check_time < latest.check_time )
(The same indexes will be fine.)
The query seems to be getting "all but the latest"; is that what you wanted?
Here is my posts table:
CREATE TABLE `posts` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`date` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`img` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`vid` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`title` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`subtitle` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`auth` varchar(54) COLLATE utf8_croatian_ci NOT NULL,
`story` longtext COLLATE utf8_croatian_ci NOT NULL,
`tags` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`status` varchar(100) COLLATE utf8_croatian_ci NOT NULL,
`moder` varchar(50) COLLATE utf8_croatian_ci NOT NULL,
`rec` varchar(50) COLLATE utf8_croatian_ci NOT NULL,
`pos` varchar(50) COLLATE utf8_croatian_ci NOT NULL,
`inde` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=117 DEFAULT CHARSET=utf8 COLLATE=utf8_croatian_ci
I want to make two partitions in order to improve query performances.
First partition should contain all non-archive rows.
Second partition - all archive rows.
ALTER TABLE posts
PARTITION BY LIST COLUMNS (status)
(
PARTITION P1 VALUES IN ('admin', 'moder', 'public', 'rec'),
PARTITION P2 VALUES IN ('archive')
);
phpmyadmin error:
Static analysis:
1 errors were found during analysis.
Unrecognized alter operation. (near "" at position 0)
MySQL said:
#1503 - A PRIMARY KEY must include all columns in the table's partitioning function
Any help?
What queries are you trying to speed up? Since the only index you currently have, WHERE id=... or WHERE id BETWEEN ... AND ... are the only queries that will be fast. And the partitioning you suggest will not help much for other queries.
You seem to have only dozens of rows; don't consider partitioning unless you expect to have at least a million rows.
status has only 5 values? Then make it ENUM('archive', 'admin', 'moder', 'public', 'rec') NOT NULL. That will take 1 byte instead of lots.
If you will be querying on date and/or status and/or auth, then let's talk about indexes, especially 'composite' indexes on such. And, to achieve the "archive" split you envision, put status as the first column in the index.
trying to query a large table (senddb.order_histories) that has close to 50M rows and this is the MySQL query I am using:
FIRST APPROACH- inner join:
select a.id,
a.order_number,
a.sku_id,
a.fulfillment_status,
a.modified_by,
a.created_at,
a.updated_at
from senddb.order_line_items a
inner join (
select order_line_item_id,
order_number,
order_status,
order_status_description,
action,
modified_by,
created_at,
max(updated_at) as updated_at
from senddb.order_histories
where order_status in ('x','y','z')
and fulfillment_location = 'abcd'
group by order_line_item_id) as b
on a.id = b.order_line_item_id
and a.fulfillment_status = '2';
EXPLAIN output :
SECOND APPROACH- nested select:
select a.id,
a.order_number,
a.sku_id,
a.fulfillment_status,
a.modified_by,
a.created_at,
a.updated_at
from senddb.order_line_items a
where a.fulfillment_status = '2'
and a.id in (
select b.order_line_item_id from(
select order_line_item_id,
order_number,
order_status,
order_status_description,
action,
modified_by,
created_at,
max(updated_at) as updated_at
from senddb.order_histories
where
order_status in ('x','y','z')
and fulfillment_location = 'abcd'
group by order_line_item_id) as b);
I believe nested select is a bad approach on large data but i anyhow added it here because it worked on my sample set. Anyway both the queries eventually time out after 600 seconds with the message : Error Code: 2013. Lost connection to MySQL server during query.
I would like to know if there are any ways to alter the query to make it run faster. I have already tried reducing the columns in the inner select / inner join but that should not really be an issue IMO. I also looked up a solution that says "create a clustered index" but i wasn't really able to follow. Any help is appreciated.
TABLE order_histories :
order_histories CREATE TABLE `order_histories` (
`id` int(4) unsigned NOT NULL AUTO_INCREMENT,
`order_number` varchar(24) DEFAULT NULL,
`order_status_description` varchar(255) DEFAULT NULL,
`datetime_stamp` datetime DEFAULT NULL,
`action` varchar(32) DEFAULT NULL,
`fulfillment_location` int(8) DEFAULT NULL,
`order_status` int(8) DEFAULT NULL,
`user_id` int(8) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
`modified_by` varchar(32) DEFAULT NULL,
`order_line_item_id` int(11) DEFAULT NULL,
`pooled` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
KEY `order_histories_ecash_idx` (`order_number`),
KEY `order_line_item_id` (`order_line_item_id`)
) ENGINE=InnoDB AUTO_INCREMENT=454738178 DEFAULT CHARSET=latin1
TABLE order_line_items :
order_line_items CREATE TABLE `order_line_items` (
`id` int(4) unsigned NOT NULL AUTO_INCREMENT,
`order_number` varchar(24) DEFAULT NULL,
`sku_id` int(8) DEFAULT NULL,
`original_price` float DEFAULT NULL,
`dept_description` varchar(100) DEFAULT NULL,
`description` varchar(100) DEFAULT NULL,
`quantity_ordered` int(8) DEFAULT NULL,
`gift_indicator` char(1) DEFAULT NULL,
`gift_wrap_flag` char(1) DEFAULT NULL,
`shipping_record_flag` char(1) DEFAULT NULL,
`gift_comments` varchar(100) DEFAULT NULL,
`item_status` char(1) DEFAULT NULL,
`tax_amount` float DEFAULT NULL,
`tax_rate` float DEFAULT NULL,
`upc` varchar(20) DEFAULT NULL,
`final_price` float DEFAULT NULL,
`line_number` int(8) DEFAULT NULL,
`master_line_number` int(8) DEFAULT NULL,
`gift_wrap_flag_type` char(1) DEFAULT NULL,
`color_code` varchar(4) DEFAULT NULL,
`size_id` varchar(6) DEFAULT NULL,
`width_id` varchar(6) DEFAULT NULL,
`brand` varchar(15) DEFAULT NULL,
`vpn` varchar(30) DEFAULT NULL,
`dept_number` int(8) DEFAULT NULL,
`class_number` int(8) DEFAULT NULL,
`non_merch_item` char(1) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
`modified_by` varchar(32) DEFAULT NULL,
`chain_id` int(11) DEFAULT NULL,
`fulfillment_location` int(11) DEFAULT NULL,
`fulfillment_date` datetime DEFAULT NULL,
`fulfillment_status` int(11) DEFAULT NULL,
`fulfillment_sales_associate` int(11) DEFAULT NULL,
`gift_wrap_line_number` int(11) DEFAULT NULL,
`shipping_type` int(11) DEFAULT NULL,
`order_track_info_id` int(11) DEFAULT NULL,
`store_tlog_updated` varchar(1) DEFAULT NULL,
`shipping_tlx_code` int(11) DEFAULT NULL,
`store_closed` tinyint(1) DEFAULT NULL,
`flags` int(11) DEFAULT NULL,
`deal_based_index` int(11) DEFAULT NULL,
`tlog_calc_ret_price` float DEFAULT NULL,
`tlog_amount` float DEFAULT NULL,
`tlog_retail_price` float DEFAULT NULL,
`tlog_ext_amount` float DEFAULT NULL,
`tlog_flag_1` int(11) DEFAULT NULL,
`tlog_flag_2` int(11) DEFAULT NULL,
`tlog_flag_3` int(11) DEFAULT NULL,
`time_remaining` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `order_line_items_ecash_idx` (`order_number`),
KEY `order_line_item_fulfillment_location_idx` (`fulfillment_location`),
KEY `order_line_item_fulfillment_status_idx` (`fulfillment_status`),
KEY `upc_idx` (`upc`),
KEY `sku_id_idx` (`sku_id`),
KEY `order_line_items_idx001` (`order_number`,`id`,`fulfillment_status`),
KEY `order_track_info_id` (`order_track_info_id`),
KEY `shipping_type_idx` (`shipping_type`,`non_merch_item`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=11367052 DEFAULT CHARSET=latin1
This query can be simplified:
select a.id,
a.order_number,
a.sku_id,
a.fulfillment_status,
a.modified_by,
a.created_at,
a.updated_at
from senddb.order_line_items a
inner join senddb.order_histories b on a.id = b.order_line_item_id
where b.order_status in ('x','y','z')
and b.fulfillment_location = 'abcd'
and a.fulfillment_status = '2';
Since you're only selecting values from table a, you don't need to select specific values from table b and can instead just apply your conditions. Outside of this, you need to ensure that b.order_line_item_id has an index on it. You can find more about creating indexes here. I'm not an expert in MySQL but something similar to this should work if senddb.order_histories.order_line_item_id isn't already the primary key.
CREATE INDEX IX_order_histories_order_line_item_id ON order_histories (order_line_item_id);
You need to read up the optimization section of the MySQL docs. It contains a lot of information on how you can optimize your queries and data sets. The main idea here is to add indexes to the fields that are being used as the criteria in the WHERE clause of the SQL statements.
Basically, both of your alternatives are using a "sub-SELECT, not an INNER JOIN.
The syntax of a true JOIN is one of the following:
SELECT ...
FROM X INNER JOIN Y USING (field_list)
... or ...
SELECT ...
FROM X INNER JOIN Y ON (x.field1 = y.field2) ...
But in both cases the objects being joined are tables or views.
I'm going to presume ... admittedly, without checking ... that Nick Larsen's answer #1 adequately re-expresses your original query using JOINs.
(Notice how, in his answer, the shorthand identifiers A and B are introduced as referring to each of the two table-names mentioned in his query.)
Firstly, you need to decide if a 50 million resultset is what you are asking for. Big data tables are not there so that you can select all their rows. They are there so that you can ask them questions using sql queries. SQL is a query language, it's not a data loading language.
What's your purpose? If you want to copy the data you can do that by loading the data, for example, 1000 rows per query in a for loop. if you are loading the data for processing, you can do that in the same way.
If you want to derive statistical information, you can use outer join and return a low number of rows, using aggregate functions. But you shouldn't do that either, what you "should" do is to decide what you want from the table and preferably, run aggregate functions to store useful information in a different table. (mostly SELECT INTO queries) You should never need to join a table of 50 million records in the first place.
Telling you how to do something wrong using indexes wouldn't be the right thing here.
I ran this sql query in my database:
update payments set method = 'paysafecard' AND amount = 25 WHERE payment_id IN (1,2,3,4,5,...)
Of course i meant set method = 'paysafecard' , amount = 25
However I did it in phpmyadmin and it showed me that rows were affected. After running it again it showed 0 rows affected.
I don't know what may have changed in the database, what could this have done?
My table looks like this:
CREATE TABLE IF NOT EXISTS `payments` (
`payment_id` int(11) NOT NULL AUTO_INCREMENT,
`method_unique_id` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`method` enum('moneybookers','paypal','admin','wallet','voucher','sofortueberweisung','bitcoin','paysafecard','paymentwall') COLLATE utf8_unicode_ci NOT NULL,
`method_tid` int(11) DEFAULT NULL,
`uid` int(11) NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`plan` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`expires_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`amount` decimal(8,2) NOT NULL,
`currency` enum('EUR','USD','BTC') COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`payment_id`),
UNIQUE KEY `method` (`method`,`method_tid`),
UNIQUE KEY `method_unique_id` (`method_unique_id`,`method`),
KEY `expires_at` (`expires_at`),
KEY `uid` (`uid`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=8030 ;
I am running
-- Server version: 5.1.41
-- PHP Version: 5.3.2-1ubuntu4.11
This would result in the method field being set to '0' for all of your records fitting the where clause.
It is interpreted as the following:
set method = ('paysafecard' AND amount = 25)
This is a logical AND, and results in a boolean value for these records(which will be parsed to the corresponding field of your column).