SQL delete query from Table - mysql

Hello everyone I want to remove any redundancy of lines in the circulaire table with the same code product(code_prd) by year (annee), object(objet), sector(secteur), circular number ( num_circulaire )
except product code equal to "-"
sql code of the table
CREATE TABLE `circulaire` (
`id` int(11) NOT NULL,
`code_prd` varchar(255) DEFAULT NULL,
`num_circulaire` double DEFAULT NULL,
`annee` double DEFAULT NULL,
`date` datetime DEFAULT NULL,
`objet` double DEFAULT NULL,
`libelle_prd` varchar(100) DEFAULT NULL,
`secteur` varchar(1) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

This looks like MySQL. You can use delete with join:
delete c
from circulaire c left join
(select code_prd, annee, objet, secteur, num_circulaire, min(id) as min_id
from circulaire
group by code_prd, annee, objet, secteur, num_circulaire
) cc
on c.id = cc.min_id
where c.code_prd <> '-' and
cc.min_id is null;
This calculates the minimum id for the combination of columns you have specified. The LEFT JOIN finds matches on the minimum id for those columns. Only non-matches are deleted.

Try this:
DELETE
FROM
circulaire
WHERE
id IN(
SELECT
*
FROM
(
SELECT
MIN(id)
FROM
circulaire
GROUP BY
code_prd, annee, objet, secteur, num_circulaire
HAVING
COUNT(*) > 1
) temp
)

Related

how to count items inside group_concat method with mysql query

I have one to many table relationship :
one user for multiple event
one event for multiple event_attribute
Now, I group by userId and want to know how many for each event attribute ?
I am using group_concat like this:
group_concat(
concat(event_event_attribute.event_attr_id,
count( distinct event_event_attribute.value)
) group by event_attr_id)
)
group by userId
So here, I first group by userId, then group concat event-attribute, at least I hope to have :
(attr1, 10),(attr2, 30)....
all in one row.
But this does not work at all
Any suggestions?
To be more specific, this is the DB schema I am using:
CREATE TABLE `user` (
`id` int(11) NOT NULL,
`name` varchar(45) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `id_UNIQUE` (`id`)
);
CREATE TABLE `event` (
`id` int(11) NOT NULL,
`name` varchar(45) DEFAULT NULL,
`user_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`)
);
CREATE TABLE `event_attr` (
`id` int(11) NOT NULL,
`att_name` varchar(45) DEFAULT NULL,
`event_id` varchar(45) DEFAULT NULL,
PRIMARY KEY (`id`)
);
INSERT INTO `user` VALUES (1,'user1'),(2,'user2'),(3,'user3');
INSERT INTO `event` VALUES (1,'event1',1),(2,'event2',1),(3,'event3',1),(4,'event4',2),(5,'event5',2),(6,'event6',3);
INSERT INTO `event_attr` VALUES (1,'att1','1'),(2,'att2','1'),(3,'att3','1'),(4,'att1','2'),(5,'att2',NULL);
Now if I am running:
select u.id, group_concat(e.name)
from user u
join event e on u.id=e.user_id
group by u.id
I will get:
1 event1,event2,event3
2 event4,event6
3 event 6
That is fine. But one step forward, I need to know count for each event_attt for each user, such as:
1 evet_att1:3;event_att2:2
2 event_att3:1
Then it is not possible. Can I use just one query to get above expected response?
should be the inverse alias concat the aggreagted values and not aggregated the concat
select concat (group_concat(event_event_attribute.event_attr_id )
,' - ',
count( distinct event_event_attribute.value) )
from event_event_attribute
group by userid
Otherwise could be you need an subquery for obtain the count group by event_attr_id
select group_concat(
concat(event_attr_id), ',', count_value)
)
from t (
select user_id, event_event_attribute.event_attr_id, count( distinct event_event_attribute.value) count_value
from event_event_attribute
group by event_attr_id
) t
group by user_id

How to use SET with SELECT DISTINCT

I have a table:
CREATE TABLE `test` (
`id` int(11) NOT NULL,
`pn` varchar(40) NOT NULL,
`price` int(1) NOT NULL,
`company` varchar(30) NOT NULL,
`flag` varchar(1) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
What I want to do is to find cheapest goods (pn) and I've managed to do it:
SELECT DISTINCT min(price), pn, company FROM `test` GROUP BY pn
But how I can also mark cheapest goods with flag. I want for all the results Update tabel, set flag=1.
How to do that? Is it possible to use UPDATE table with SELECT DISTINCT?
Here is a SQL Fiddle:
http://sqlfiddle.com/#!9/ea1b3f/9
First, select distinct with group by is almost never the right thing to do. Presumably, you intend something like:
SELECT min(price), pn, min(company)
FROM `test`
GROUP BY pn;
If you want to set a flag for all the min prices, use a join:
update test t join
(select pn, min(price) as minprice
from test
group by pn
) tt
on t.pn = tt.pn and t.price = tt.minprice
set t.flag = 1;

Improve speed of MySQL query with 5 left joins

Working on a support ticketing system with not a lot of tickets (~3,000). To get a summary grid of ticket information, there are five LEFT JOIN statements on custom field table (j25_field_value) containing about 10,000 records. The query runs too long (~10 seconds) and in cases with a WHERE clause, it runs even longer (up to ~30 seconds or more).
Any suggestions for improving the query to reduce the time to run?
Four tables:
j25_support_tickets
CREATE TABLE `j25_support_tickets` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`category_id` int(11) NOT NULL DEFAULT '0',
`user_id` int(11) DEFAULT NULL,
`email` varchar(50) DEFAULT NULL,
`subject` varchar(255) DEFAULT NULL,
`message` text,
`modified_date` datetime DEFAULT NULL,
`priority_id` tinyint(3) unsigned DEFAULT NULL,
`status_id` tinyint(3) unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=3868 DEFAULT CHARSET=utf8
j25_support_priorities
CREATE TABLE `j25_support_priorities` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`title` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=14 DEFAULT CHARSET=utf8
j25_support_statuses
CREATE TABLE `j25_support_statuses` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`title` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=7 DEFAULT CHARSET=utf8
j25_field_value (id, ticket_id, field_id, field_value)
CREATE TABLE `j25_support_field_value` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`ticket_id` int(11) DEFAULT NULL,
`field_id` int(11) DEFAULT NULL,
`field_value` tinytext,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=10889 DEFAULT CHARSET=utf8
Also, ran this:
SELECT LENGTH(field_value) len FROM j25_support_field_value ORDER BY len DESC LIMIT 1
note: the result = 38
The query:
SELECT DISTINCT t.id as ID
, (select p.title from j25_support_priorities p where p.id = t.priority_id) as Priority
, (select s.title from j25_support_statuses s where s.id = t.status_id) as Status
, t.subject as Subject
, t.email as SubmittedByEmail
, type.field_value AS IssueType
, ver.field_value AS Version
, utype.field_value AS UserType
, cust.field_value AS Company
, refno.field_value AS RefNo
, t.modified_date as Modified
FROM j25_support_tickets AS t
LEFT JOIN j25_support_field_value AS type ON t.id = type.ticket_id AND type.field_id =1
LEFT JOIN j25_support_field_value AS ver ON t.id = ver.ticket_id AND ver.field_id =2
LEFT JOIN j25_support_field_value AS utype ON t.id = utype.ticket_id AND utype.field_id =3
LEFT JOIN j25_support_field_value AS cust ON t.id = cust.ticket_id AND cust.field_id =4
LEFT JOIN j25_support_field_value AS refno ON t.id = refno.ticket_id AND refno.field_id =5
ALTER TABLE j25_support_field_value
ADD INDEX (`ticket_id`,`field_id`,`field_value`(50))
This index will work as a covering index for your query. It will allow the joins to use only this index to look up the values. It should perform massively faster than without this index, since currently your query would have to read every row in the table to find what matches each combination of ticket_id and field_id.
I would also suggest converting your tables to InnoDB engine, unless you have a very explicit reason for using MyISAM.
ALTER TABLE tablename ENGINE=InnoDB
As above - a better index would help. You could probably then simplify your query into something like this too (join to the table only once):
SELECT t.id as ID
, p.title as Priority
, s.title as Status
, t.subject as Subject
, t.email as SubmittedByEmail
, case when v.field_id=1 then v.field_value else null end as IssueType
, case when v.field_id=2 then v.field_value else null end as Version
, case when v.field_id=3 then v.field_value else null end as UserType
, case when v.field_id=4 then v.field_value else null end as Company
, case when v.field_id=5 then v.field_value else null end as RefNo
, t.modified_date as Modified
FROM j25_support_tickets AS t
LEFT JOIN j25_support_field_value v ON t.id = v.ticket_id
LEFT JOIN j25_support_priorities p ON p.id = t.priority_id
LEFT JOIN j25_support_statuses s ON s.id = t.status_id;
You can do away with the subqueries for starters and just get them from another join. You can add an index to j25_support_field_value
alter table j25_support_field_value add key(id, field_type);
I assume there is an index on id in j25_support_tickets - if not and if they are unique, add a unique index alter table j25_support_tickets add unique key(id); If they're not unique, remove the word unique from that statement.
In MySQL, a join usually requires an index on the field(s) that you are using to join on. This will hold up and produce very reasonable results with huge tables (100m+), if you follow that rule, you will not go wrong.
are the ids in j25_support_tickets unique? If they are you can do away with the distinct - if not, or if you are getting exact dupicates in each row, still do away with the distinct and add a group by t.id to the end of this:
SELECT t.id as ID
, p.title as Priority
, s.title as Status
, t.subject as Subject
, t.email as SubmittedByEmail
, type.field_value AS IssueType
, ver.field_value AS Version
, utype.field_value AS UserType
, cust.field_value AS Company
, refno.field_value AS RefNo
, t.modified_date as Modified
FROM j25_support_tickets AS t
LEFT JOIN j25_support_field_value AS type ON t.id = type.ticket_id AND type.field_id =1
LEFT JOIN j25_support_field_value AS ver ON t.id = ver.ticket_id AND ver.field_id =2
LEFT JOIN j25_support_field_value AS utype ON t.id = utype.ticket_id AND utype.field_id =3
LEFT JOIN j25_support_field_value AS cust ON t.id = cust.ticket_id AND cust.field_id =4
LEFT JOIN j25_support_field_value AS refno ON t.id = refno.ticket_id AND refno.field_id =5
LEFT JOIN j25_support_priorities p ON p.id = t.priority_id
LEFT JOIN j25_support_statuses s ON s.id = t.status_id;
Switch to InnoDB.
After switching to InnoDB, make the PRIMARY KEY for j25_support_field_value be (ticket_id, field_id) (and get rid if id). (Tacking on field_value(50) will hurt, not help.)
A PRIMARY KEY is a UNIQUE KEY, so don't have both.
Use VARCHAR(255) instead of the nearly-equivalent TINYTEXT.
EAV schema sucks. My ran on EAV.

MysqL big table query optimization

I have a chatting application. I have an api which returns list of users who the user talked. But it takes a long to mysql return a list messages when it reachs 100000 rows of data.
This is my messages table
CREATE TABLE IF NOT EXISTS `messages` (
`_id` int(11) NOT NULL AUTO_INCREMENT,
`fromid` int(11) NOT NULL,
`toid` int(11) NOT NULL,
`message` text NOT NULL,
`attachments` text NOT NULL,
`status` tinyint(1) NOT NULL DEFAULT '0',
`date` datetime NOT NULL,
`delete` varchar(50) NOT NULL,
`uuid_read` varchar(250) NOT NULL,
PRIMARY KEY (`_id`),
KEY `fromid` (`fromid`,`toid`,`status`,`delete`,`uuid_read`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=118561 ;
and this is my users table (simplified)
CREATE TABLE IF NOT EXISTS `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`login` varchar(50) DEFAULT '',
`sex` tinyint(1) DEFAULT '0',
`status` varchar(255) DEFAULT '',
`avatar` varchar(30) DEFAULT '0',
`last_active` datetime DEFAULT NULL,
`active` tinyint(1) DEFAULT '1',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=15523 ;
And here is my query (for user with id 1930)
select SQL_CALC_FOUND_ROWS `u_id`, `id`, `login`, `sex`, `birthdate`, `avatar`, `online_status`, SUM(`count`) as `count`, SUM(`nr_count`) as `nr_count`, `date`, `last_mesg` from
(
(select `m`.`fromid` as `u_id`, `u`.`id`, `u`.`login`, `u`.`sex`, `u`.`birthdate`, `u`.`avatar`, `u`.`last_active` as online_status, COUNT(`m`.`_id`) as `count`, (COUNT(`m`.`_id`)-SUM(`m`.`status`)) as `nr_count`, `tm`.`date` as `date`, `tm`.`message` as `last_mesg` from `messages` as m inner join `messages` as tm on `tm`.`_id`=(select MAX(`_id`) from `messages` as `tmz` where `tmz`.`fromid`=`m`.`fromid`) left join `users` as u on `u`.`id`=`m`.`fromid` where `m`.`toid`=1930 and `m`.`delete` not like '%1930;%' group by `u`.`id`)
UNION
(select `m`.toid as `u_id`, `u`.`id`, `u`.`login`, `u`.`sex`, `u`.`birthdate`, `u`.`avatar`, `u`.`last_active` as online_status, COUNT(`m`.`_id`) as `count`, 0 as `nr_count`, `tm`.`date` as `date`, `tm`.`message` as `last_mesg` from `messages` as m inner join `messages` as tm on `tm`.`_id`=(select MAX(`_id`) from `messages` as `tmz` where `tmz`.`toid`=`m`.`toid`) left join `users` as u on `u`.`id`=`m`.`toid` where `m`.`fromid`=1930 and `m`.`delete` not like '%1930;%' group by `u`.`id`)
order by `date` desc ) as `f` group by `u_id` order by `date` desc limit 0,10
Please help to optimize this query
What I need,
Who user talked to (name, sex, and etc)
What was the last message (from me or to me)
Count of messages (all)
Count of unread messages (only to me)
The query works well, but takes too long.
The output must be like this
You have some design problems on your query and database.
You should avoid keywords as column names, as that delete column or the count column;
You should avoid selecting columns not declared in the group by without an aggregation function... although MySQL allows this, it's not a standard and you don't have any control on what data will be selected;
Your not like construction may cause a bad behavior on your query because '%1930;%' may match 11930; and 11930 is not equal to 1930;
You should avoid like constructions starting and ending with % wildcard, which will cause the text processing to take longer;
You should design a better way to represent a message deletion, probably a better flag and/or another table to save any important data related with the action;
Try to limit your result before the join conditions (with a derived table) to perform less processing;
I tried to rewrite your query the best way I understood it. I've executed my query in a messages table with ~200.000 rows and no indexes and it performed in 0,15 seconds. But, for sure you should create the right indexes to help it perform better when the amount of data increase.
SELECT SQL_CALC_FOUND_ROWS
u.id,
u.login,
u.sex,
u.birthdate,
u.avatar,
u.last_active AS online_status,
g._count,
CASE WHEN m.toid = 1930
THEN g.nr_count
ELSE 0
END AS nr_count,
m.`date`,
m.message AS last_mesg
FROM
(
SELECT
MAX(_id) AS _id,
COUNT(*) AS _count,
COUNT(*) - SUM(m.status) AS nr_count
FROM messages m
WHERE 1=1
AND m.`delete` NOT LIKE '%1930;%'
AND
(0=1
OR m.fromid = 1930
OR m.toid = 1930
)
GROUP BY
CASE WHEN m.fromid = 1930
THEN m.toid
ELSE m.fromid
END
ORDER BY MAX(`date`) DESC
LIMIT 0, 10
) g
INNER JOIN messages AS m ON 1=1
AND m._id = g._id
LEFT JOIN users AS u ON 0=1
OR (m.fromid <> 1930 AND u.id = m.fromid)
OR (m.toid <> 1930 AND u.id = m.toid)
ORDER BY m.`date` DESC
;

Best way to search two queries and eliminate rows without a relationship

I am working on a property website and have record sets for property and for unit, unit has a one-to-many relationship with property. What I'm trying to figure out is how to best create a search function which will output results based on criteria from both. So if I search for a property with the location Manchester and a unit with a freehold tenure I'd like to eliminate all properties which don't have a unit with the tenure of freehold.
A potential solution I've considered is to create a record set for properties which match the property criteria and then create a unit record set for units which match the unit criteria and then finally loop through the property record set in server-side code and eliminate any properties which aren't related to any of the units in the unit record set. Really not sure if this is the best way to do things though so would be keen to hear any suggestions?
Thanks
EDIT (Added table structure and MySQL):
--
-- Table structure for table `property`
--
CREATE TABLE IF NOT EXISTS `property` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` text NOT NULL,
`street` text NOT NULL,
`town` text NOT NULL,
`postcode` text NOT NULL,
`description` longtext NOT NULL,
`team_member` varchar(255) NOT NULL DEFAULT '',
`pdf` text NOT NULL,
`default_image_id` int(11) DEFAULT NULL,
`virtual_tour_link` text NOT NULL,
`date` date NOT NULL DEFAULT '0000-00-00',
`archive` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='' AUTO_INCREMENT=13 ;
--
-- Table structure for table `unit`
--
CREATE TABLE IF NOT EXISTS `unit` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` text NOT NULL,
`description` text NOT NULL,
`size_sq_ft` int(11) DEFAULT NULL,
`size_acres` float DEFAULT NULL,
`price` float DEFAULT NULL,
`rental_price` float DEFAULT NULL,
`on_application` tinyint(1) DEFAULT NULL,
UNIQUE KEY `id` (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='Stores data for property units' AUTO_INCREMENT=5;
--
-- Table structure for table `property_to_unit`
--
CREATE TABLE IF NOT EXISTS `property_to_unit` (
`property_id` int(11) NOT NULL,
`unit_id` int(11) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
--
-- MySQL which produces list of properties
--
SELECT
P.id AS id,
P.name AS name,
P.street AS street,
P.town AS town,
P.postcode AS postcode,
P.description AS description,
P.team_member AS team_member,
P.pdf AS pdf,
P.virtual_tour_link AS virtual_tour_link,
P.date AS date,
P.archive AS archive,
PI.name as image,
P2.image_ids as image_ids,
L2.location_ids as location_ids,
U2.unit_ids as unit_ids
FROM property P
-- Get default image and join using property id
LEFT JOIN property_image PI ON PI.id = P.default_image_id
-- Create a list of image_ids from property_image and
-- property_to_property_image tables then join using property_id
LEFT JOIN (
SELECT
property_id,
GROUP_CONCAT(CAST(id AS CHAR)) as image_ids
FROM property_to_property_image PTPI
LEFT JOIN property_image PI ON PI.id = PTPI.property_image_id
GROUP BY property_id
) P2 ON P2.property_id = P.id
-- Create a list of locations from property_location table
-- and join using property_id
LEFT JOIN (
SELECT
property_id,
property_location_id,
GROUP_CONCAT(CAST(property_location.id AS CHAR)) AS location_ids
FROM property_to_property_location
INNER JOIN property_location ON property_location.id = property_to_property_location.property_location_id
GROUP BY property_id
) L2 ON L2.property_id = P.id
-- Create a list of units from unit table
-- and join using property_id
LEFT JOIN (
SELECT
property_id,
unit_id,
GROUP_CONCAT(CAST(unit_id AS CHAR)) AS unit_ids
FROM property_to_unit
INNER JOIN unit ON unit.id = property_to_unit.unit_id
GROUP BY property_id
) U2 ON U2.property_id = P.id
--
-- MySQL which produces list of units
--
SELECT
id,
name,
description,
size_sq_ft,
size_acres,
price,
rental_price,
on_application,
tenure_ids,
tenure_names,
type_ids,
type_names
FROM unit AS U
-- join tenure ids and names
LEFT JOIN (
SELECT
unit_id,
GROUP_CONCAT( CAST(UT.id AS CHAR) ) AS tenure_ids,
GROUP_CONCAT(UT.name) AS tenure_names
FROM unit_to_unit_tenure UTUT
INNER JOIN unit_tenure UT ON UT.id = UTUT.unit_tenure_id
GROUP BY unit_id
) UT ON UT.unit_id = U.id
-- join type ids and names
LEFT JOIN (
SELECT
unit_id,
GROUP_CONCAT( CAST(UTYPE.id AS CHAR) ) AS type_ids,
GROUP_CONCAT(UTYPE.name) AS type_names
FROM unit_to_unit_type UTUT
INNER JOIN unit_type UTYPE ON UTYPE.id = UTUT.unit_type_id
GROUP BY unit_id
) UTYPE ON UTYPE.unit_id = U.id
WHERE 0=0
I'm currently using a dynamically created WHERE statement appended to each MySQL query to filter the property and unit results.
You're making it a bit more complicated than it is. If I understand correctly, you can easily do this in a single query. This would search properties that have units with a particlar unit tenure id:
select *
from property p
where p.id in (
select pu.property_id
from property_to_unit pu
inner join unit u ON pu.unit_id = u.id
inner join unit_to_unit_tenure uut ON u.id = uut.unit_id
where uut.id = <cfqueryparam value="#uutid#">
)
Using two queries and then looping through to cross-check sounds like it could be dog slow.
Your situation requires a posted foreign key in the property table. Store the unit_id in the property table and use a join in your query such as:
select * from property p, unit u
where p.unit_id = u.id
and p.town = ....
EDIT: So I just noticed the rest of your SQL. If you require to keep the many-to-many relationship table for the unit -> property relationship then you will need to join unit and property off of that table.