the code below is work but it take long time to run sometime the php code stop but it work well with mysql graphical tool (becase it can wait) . Are there any method to make it faster I use index in some column it help Are there any method to create table if I create table i think it will improve performance but this table it must be update maybe one a week what is the good practice please suggest
SELECT member.own, member.provincecode, province.PROVINCE_NAME, member.amphurecode,
amphur.AMPHUR_NAME, member.Sname, member.Ssurname, member.Hno, member.Moo,
member.Sex, member.tambol, member.dateofbirth, member.migratedate,
Year( Current_Date( ) ) - Year( member.dateofbirth ) AS y,
Year( Current_Date( ) ) - Year( member.migratedate ) AS m
FROM member
LEFT JOIN amphur ON ( member.amphurecode
COLLATE utf8_general_ci = amphur.AMPHUR_CODE )
LEFT JOIN province ON member.provincecode
COLLATE utf8_general_ci = province.PROVINCE_CODE
Collate is an expensive operation. So assuming that member has more rows thanprovince, try to collate the table with the smaller number of rows:
LEFT JOIN province ON member.provincecode
= province.PROVINCE_CODE COLLATE <collation of member.provincecode>
It would be even better to give all columns in all tables the same collation.
Related
NOTE : I tried many SF solution, but none work for me. This is bit challenging for, any help will be appreciated.
Below is my SQL-Fiddle link : http://sqlfiddle.com/#!9/6daa20/9
I have tables below:
CREATE TABLE `tbl_pay_chat` (
nId int(11) NOT NULL AUTO_INCREMENT,
npayid int(11) NOT NULL,
nSender int(11) NOT NULL,
nTos varchar(255) binary DEFAULT NULL,
nCcs varchar(255) binary DEFAULT NULL,
sMailBody varchar(500) binary DEFAULT NULL,
PRIMARY KEY (nId)
)
ENGINE = INNODB,
CHARACTER SET utf8,
COLLATE utf8_bin;
INSERT INTO tbl_pay_chat
(nId,npayid,nSender,nTos,nCcs,sMailBody)
VALUES
(0,1,66,'3,10','98,133,10053','Hi this test maail'),
(0,1,66,'3,10','98,133,10053','test mail received');
_____________________________________________________________
CREATE TABLE `tbl_emp` (
empid int(11) NOT NULL,
fullname varchar(45) NOT NULL,
PRIMARY KEY (empid)
)
ENGINE = INNODB,
CHARACTER SET utf8,
COLLATE utf8_bin;
INSERT INTO `tbl_emp` (empid,fullname)
VALUES
(3, 'Rio'),
(10, 'Christ'),
(66, 'Jack'),
(98, 'Jude'),
(133, 'Mike'),
(10053, 'James');
What I want :
JOIN above two tables to get fullname in (nTos & nCcs) columns.
Also, I want total COUNT() of rows.
What I tried is below query but getting multiples time FULLNAME in 'nTos and nCcs column' also please suggest to find proper number of row count.
SELECT a.nId, a.npayid, e1.fullname AS nSender, sMailBody, GROUP_CONCAT(b.fullname ORDER BY b.empid)
AS nTos, GROUP_CONCAT(e.fullname ORDER BY e.empid) AS nCcs
FROM tbl_pay_chat a
INNER JOIN tbl_emp b
ON FIND_IN_SET(b.empid, a.nTos) > 0
INNER JOIN tbl_emp e
ON FIND_IN_SET(e.empid, a.nCcs) > 0
JOIN tbl_emp e1
ON e1.empid = a.nSender
GROUP BY a.nId ORDER BY a.nId DESC;
I hope I made my point clear. Please help.
You have a horrible data model. You should not be storing lists of ids in strings. Why? Here are some reasons:
Numbers should be stored as numbers not strings.
Relationships between tables should be declared using foreign key relationships.
SQL has pretty poor string manipulation capabilities.
The use of functions and type conversion in ON often prevents the use of indexes.
No doubt there are other good reasons. Your data model should be using properly declared junction tables for the n-m relationships.
That said, sometimes we are stuck with other people's really, really, really, really bad design decisions. There are some ways around this. I think the query that you want can be expressed as:
SELECT pc.nId, pc.npayid, s_e.fullname AS nSender, pc.sMailBody,
GROUP_CONCAT(DISTINCT to_e.fullname ORDER BY to_e.empid)
AS nTos,
GROUP_CONCAT(DISTINCT cc_e.fullname ORDER BY cc_e.empid) AS nCcs
FROM tbl_pay_chat pc INNER JOIN
tbl_emp to_e
ON FIND_IN_SET(to_e.empid, pc.nTos) > 0 INNER JOIN
tbl_emp cc_e
ON FIND_IN_SET(cc_e.empid, pc.nCcs) > 0 JOIN
tbl_emp s_e
ON s_e.empid = pc.nSender
GROUP BY pc.nId
ORDER BY pc.nId DESC;
Here is a db<>fiddle.
I have a MySQL table that looks (very simplified) like this:
CREATE TABLE `logging` (
`id` bigint(20) NOT NULL,
`time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`level` smallint(3) NOT NULL,
`message` longtext CHARACTER SET utf8 COLLATE utf8_general_mysql500_ci NOT NULL
);
I would like to delete all rows of a specific level, except the last one (time is most recent).
Is there a way to select all rows with level set to a specific value and then delete all rows except the latest one in one single SQL query? How would I start solving this problem?
(As I said, this is a very simplified table, so please don't try to discuss possible design problems of this table. I removed some columns. It is designed per PSR-3 logging standard and I don't think there is an easy way to change that. What I want to solve is how I can select from a table and then delete all but some rows of the same table. I have only intermediate knowledge of MySQL.)
Thank you for pushing me in the right direction :)
Edit:
The Database version is /usr/sbin/mysqld Ver 8.0.18-0ubuntu0.19.10.1 for Linux on x86_64 ((Ubuntu))
You can use ROW_NUMBER() analytic function ( as using DB version 8+ ) :
DELETE lg FROM `logging` AS lg
WHERE lg.`id` IN
( SELECT t.`id`
FROM
(
SELECT t.*,
ROW_NUMBER() OVER (ORDER BY `time` DESC) as rn
FROM `logging` t
-- WHERE `level` = #lvl -- optionally add this line to restrict for a spesific value of `level`
) t
WHERE t.rn > 1
)
to delete all of the rows except the last inserted one(considering id is your primary key column).
You can do this:
SELECT COUNT(time) FROM logging WHERE level=some_level INTO #TIME_COUNT;
SET #TIME_COUNT = #TIME_COUNT-1;
PREPARE STMT FROM 'DELETE FROM logging WHERE level=some_level ORDER BY time ASC LIMIT ?;';
EXECUTE STMT USING #TIME_COUNT;
If you have an AUTO_INCREMENT id column - I would use it to determine the most recent entry. Here is one way doing that:
delete l
from (
select l1.level, max(id) as id
from logging l1
where l1.level = #level
) m
join logging l
on l.level = m.level
and l.id < m.id
An index on (level) should give you good performance and will support the MAX() subquery as well as the JOIN.
View on DB Fiddle
If you really need to use the time column, you can modify the query as follows:
delete l
from (
select l1.level, l1.id
from logging l1
where l1.level = #level
order by l1.time desc, l1.id desc
limit 1
) m
join logging l
on l.level = m.level
and l.id <> m.id
View on DB Fiddle
Here you would want to have an index on (level, time).
I want to order the result of a join between 4 tables - Will creating an index for the order by column (b.SITE_ID) improve query performance?
SELECT b.SERVICE_ID, b.ATT_ID, b.SITE_ID, b.ATT_VALUE, c.KEY_NAME, d.NAME as account_name
FROM SITE_ATTRIBUTES a , SF_SITE_ATTRIBUTES b, ATTRIBUTE_DEF c,SF_SITE_MASTER d
WHERE a.SERVICE_ID=b.SERVICE_ID
and b.SERVICE_ID=c.SERVICE_ID
and b.SERVICE_ID=d.SERVICE_ID
and b.SERVICE_ID=#service_id COLLATE utf8_unicode_ci
and b.ATT_ID= c.ID
and b.ATT_ID= a.ATT_DEF
and a.SITE=b.SITE_id
and b.SITE_ID = d.ID
and a.value != b.att_value
and b.att_value is not null
ORDER BY b.SITE_ID
Thinking it won't since the order takes place over the intermediary joins result set...
b.SERVICE_ID=#service_id COLLATE utf8_unicode_ci Is inefficient since it requires changing collation on the fly. Change it so SERVICE_ID is declared COLLATE tf8_unicode_ci and your connection is using that collation. Then...
INDEX(SERVICE_ID, SITE_ID)
would probably be the best --
Filter on SERVICE_ID;
Avoid sort by having SITE_ID.
The rest of the filtering (!= and NOT NULL) is unavoidable via indexing.
Just plain INDEX(SITE_ID) is probably not as good for this query.
I found some strange(for me) behavour in MySQL. I have a simple query:
SELECT CONVERT( `text`.`old_text`
USING utf8 ) AS stext
FROM `text`
WHERE `text`.`old_id` IN
(
SELECT `revision`.`rev_text_id`
FROM `revision`
WHERE `revision`.`rev_id`
IN
(
SELECT `page_latest`
FROM `page`
WHERE `page_id` = 108
)
)
when i run it, phpmyadmin show execution time of 77.0446 seconds.
But then i replace
WHERE `text`.`old_id` IN
by
WHERE `text`.`old_id` =
it's execution time falls to about 0.001 sec. Result of this query
SELECT `revision`.`rev_text_id`
FROM `revision`
WHERE `revision`.`rev_id`
IN
(
SELECT `page_latest`
FROM `page`
WHERE `page_id` = 108
)
is
+------------+
|rev_text_id |
+------------+
|6506 |
+------------+
Can somebody please explain this behavour?
try to add INDEX on the following columns,
ALTER TABLE `text` ADD INDEX idx_text (old_id);
ALTER TABLE `revision` ADD INDEX idx_revision (rev_text_id);
and Execute the following query
SELECT DISTINCT CONVERT(a.`old_text` USING utf8 ) AS stext
FROM `text` a
INNER JOIN `revision` b
ON a.`old_id` = b.`rev_text_id`
INNER JOIN `page` c
ON b.`rev_id` = c.`page_latest`
WHERE c.`page_id` = 108
PS: Can you run also the following query and post their respective results?
DESC `text`;
DESC `revision`;
DESC `page`;
There are two primary ways you can increase your query performance here
Add Indexes (such as Kuya mentioned)
Rid yourself of the subqueries where possible
For Indexes, add an index on the columns you are searching for your matches:
text.old_id, revision.rev_text_id & page.page_id
ALTER TABLE `text` ADD INDEX idx_text (old_id);
ALTER TABLE `revision` ADD INDEX idx_revision (rev_text_id);
ALTER TABLE `page` ADD INDEX idx_page (page_id);
Your next issue is that nested-sub-selects are hell on your query execution plan. Here is a good thread discussing JOIN vs Subquery. Here is an article on how to get execution plan info from mySQL.
First looks at an execution plan can be confusing, but it will be your best friend when you have to concern yourself with query optimization.
Here is an example of your same query with just joins ( you could use inner or left and get pretty much the same result). I don't have your tables or data, so forgive synax issues (there is no way I can verify the code works verbatim in your environment, but it should give you a good starting point).
SELECT
CONVERT( `text`.`old_text` USING utf8 ) AS stext
FROM `text`
-- inner join only returns rows when it can find a
-- matching `revision`.`rev_text_id` row to `text`.`old_id`
INNER JOIN `revision`
ON `text`.`old_id` = `revision`.`rev_text_id`
-- Inner Join only returns rows when it can find a
-- matching `page_latest` row to `page_latest`
INNER JOIN `page`
ON `revision`.`rev_id` = `page`.`page_latest`
WHERE `page`.`page_id` = 108
MySQLDB is looping through each result of the inner query and comparing it with each record in the outer query.
in the second inner query;
WHERE `revision`.`rev_id`
IN
( SELECT `page_latest`
FROM `page`
WHERE `page_id` = 108
you should definitely use '=' instead of IN, since you're selecting a distinct record, there would be no point in looping through a result when you know only one record will be returned each time
I have a table with posts and I want to generate a graph that shows how many posts were made the previous last 30 minutes, and the last 30 minutes before that etc. The posts are selected by their post_handler and post_status.
The table structure looks like this.
CREATE TABLE IF NOT EXISTS `posts` (
`post_title` varchar(255) NOT NULL,
`post_content` text NOT NULL,
`post_date_added` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`post_handler` varchar(255) NOT NULL,
`post_status` tinyint(4) NOT NULL,
`id` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
KEY `post_status` (`post_status`),
KEY `post_status_2` (`post_status`,`id`),
KEY `post_handler` (`post_handler`),
KEY `post_date_added` (`post_date_added`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=2300131 ;
The results I'd like to receive, sorted after post_date_added.
period_start period_end posts
2011-12-06 19:23:44 2011-12-06 19:53:44 10
2011-12-06 19:53:44 2011-12-06 20:23:44 39
2011-12-06 20:23:44 2011-12-06 20:53:44 40
Right now I use solution where I have to run this query many times over, and then insert the data into another table from the PHP script.
SELECT COUNT(*) FROM posts WHERE post_handler = 'test' AND post_status = 1 AND post_date_added BETWEEN '2011-12-06 19:23:44' AND '2011-12-06 19:53:44'
Do you know any other solution? Is there any way to run a query that also inserts results into the database, all in one query?
Its fairly easy to group by distinctive time parameters, like hour, minute, day or whatever. If you want to group this by an hour, a possible query might look like this:
SELECT DATE_FORMAT(post_date_added,"%Y-%m-%d %H") AS "_Date",
COUNT(*)
FROM posts
WHERE post_handler = 'test'
AND post_status = 1
GROUP BY _Date;
(run this with a mysql query tool of your choice to see the output).
However, if you want to consider 30mins as the base of your group, the SQL part will get more tricky. For this special purpose, since you've only have to divide into two different subsets, maybe work with this approach:
SELECT DATE_FORMAT(post_date_added,"%Y-%m-%d %H") AS "_Date",
"00" AS "semihour",
COUNT(*)
FROM posts
WHERE post_handler = 'test'
AND DATE_FORMAT(post_date_added,"%i") < 30
AND post_status = 1
GROUP BY _Date
UNION
SELECT DATE_FORMAT(post_date_added,"%Y-%m-%d %H") AS "_Date",
"30" AS "semihour",
COUNT(*)
FROM posts
WHERE post_handler = 'test'
AND DATE_FORMAT(post_date_added,"%i") >= 30
AND post_status = 1
GROUP BY _Date;
Again, run this with a mysql query tool of your choice to see the output. You could add mathematical distinguishments there too working with CASE or IF and such, but personally I'd either group by hour or minute just to keep the SQL part way easier.
To directly add those numbers into your graph database, use this syntax:
INSERT INTO yourtable (yourfields)
SELECT ...
More details about this can be found here in the MySQL documentation.
In (very) brief: yes, you can insert the results of a query into another table. Take a look at INSERT ... SELECT here: http://dev.mysql.com/doc/refman/5.1/en/insert-select.html
Essentially, you'd just change what you have to something like
INSERT INTO post_statistics_table (period_start, period_end, posts)
SELECT ?, ?, COUNT(*) FROM posts
WHERE post_handler = 'test'
AND post_status = 1
AND post_date_added BETWEEN ? AND ?
and then fill in the four ?s with the same two DATETIMEs, repeated. ($from, $to, $from, $to)