Next and Previous ID with WHERE slow - mysql

i am trying to solve a performance issue.
I have a table called "pages" which contains thousands of pages.
There is a field called "status" which can be 0,1,2 and i want to select only 1 and 2 but not 0. As soon as i add the additional AND != 0 or IN (1,2) or >= 1 this query takes 0,5 seconds, without it its quiet fast.
Here are my queries:
SELECT name, pages.id, urlString, metaTitle, metaDescription, id,
anchorText
FROM pages
INNER JOIN metaDe
ON pages.id=metaDe.pageId
WHERE id = (SELECT min(id) FROM pages WHERE
linkToPageFunction = ? AND id > ? AND status != 0 )
LIMIT 1
SELECT name, pages.id, urlString, metaTitle,
metaDescription, id, anchorText
FROM pages
INNER JOIN metaDe
ON pages.id=metaDe.pageId
WHERE id = (SELECT max(id) FROM pages WHERE
linkToPageFunction = ? AND id < ? AND status != 0 )
LIMIT 1
without "AND status != 0" it is fast.
I want it to be fast plus if possible i want to use one query which returns two results (previous and next) instead of using two queries.
Any suggestions?
Thanks in advance
EDIT EXPLAIN:
array (
0 =>
array (
'id' => 1,
'select_type' => 'PRIMARY',
'table' => 'metaDe',
'type' => 'const',
'possible_keys' => 'PRIMARY,pageId_2,pageId,pageId_3,idxMetaDePageId',
'key' => 'PRIMARY',
'key_len' => '4',
'ref' => 'const',
'rows' => 1,
'Extra' => 'Using where',
),
1 =>
array (
'id' => 1,
'select_type' => 'PRIMARY',
'table' => 'pages',
'type' => 'const',
'possible_keys' => 'PRIMARY,id,id_2,id_3',
'key' => 'PRIMARY',
'key_len' => '4',
'ref' => 'const',
'rows' => 1,
'Extra' => 'Using where; Using index',
),
2 =>
array (
'id' => 2,
'select_type' => 'SUBQUERY',
'table' => 'pages',
'type' => 'ref',
'possible_keys' => 'PRIMARY,linkToPageFunction,id,status,id_2,id_3,idxPagesForStatusQuery,idxPagesStatus',
'key' => 'idxPagesForStatusQuery',
'key_len' => '93',
'ref' => 'const',
'rows' => 298725,
'Extra' => 'Using where; Using index',
),
)array (
0 =>
array (
'id' => 1,
'select_type' => 'PRIMARY',
'table' => 'pages',
'type' => 'const',
'possible_keys' => 'PRIMARY,id,id_2,id_3',
'key' => 'PRIMARY',
'key_len' => '4',
'ref' => 'const',
'rows' => 1,
'Extra' => 'Using index',
),
1 =>
array (
'id' => 1,
'select_type' => 'PRIMARY',
'table' => 'metaDe',
'type' => 'const',
'possible_keys' => 'PRIMARY,pageId_2,pageId,pageId_3,idxMetaDePageId',
'key' => 'PRIMARY',
'key_len' => '4',
'ref' => 'const',
'rows' => 1,
'Extra' => '',
),
2 =>
array (
'id' => 2,
'select_type' => 'SUBQUERY',
'table' => 'pages',
'type' => 'range',
'possible_keys' => 'PRIMARY,linkToPageFunction,id,status,id_2,id_3,idxPagesForStatusQuery,idxPagesStatus',
'key' => 'idxPagesForStatusQuery',
'key_len' => '97',
'ref' => NULL,
'rows' => 5,
'Extra' => 'Using where; Using index',
),
)
Since i cannot answer this thread for whatever reason, i am giving my answer and solutions here.
Thanks everybody. Sometimes you need to thing and talk about to fix it. I guess i just found the (simple) solution.
Thats what i am doing now. I have no idea why i havent got this way before.
Next:
SELECT name, pages.id, urlString, metaTitle, metaDescription, id, anchorText
FROM pages
INNER JOIN metaDe
ON pages.id=metaDe.pageId
WHERE id > ? AND status IN (1, 2) AND linkToPageFunction = ?
ORDER BY id ASC LIMIT 1
Previous:
SELECT name, pages.id, urlString, metaTitle, metaDescription, id, anchorText
FROM pages
INNER JOIN metaDe
ON pages.id=metaDe.pageId
WHERE id < ? AND status IN (1, 2) AND linkToPageFunction = ?
ORDER BY id DESC LIMIT 1

You dont seem to have an index on your pages table that includes the status field. Add such an index and your query speed should improve a lot.
create index idxPagesStatus on pages (status);
or even
create index idxPagesForStatusQuery on pages (linkToPageFunction, id, status);
to include all fields in the query. Also, on your metaDe tabke pageId could use an index as well.
create index idxMetaDePageId on metaDe (pageId);
also, if you know the pages.id key is sequential for sure(ie, goes like: 1, 2, 3, 4, 5,...), and none is deleted, you could simply do:
SELECT
name,
pages.id,
urlString,
metaTitle,
metaDescription,
id,
anchorText
FROM
pages
left JOIN
metaDe ON pages.id=metaDe.pageId
WHERE
abs(2-id)=1;

Related

Group By array value from text field using mysql

I have a table 'my_data'
Table structure
and field 'input' as type text. And I stored array value in that field as follow
array (
'msisdn' => '99999999999',
'keyword' => '',
'serviceid' => '0011001100',
'productid' => '111000111',
**'mode' => '02',**
'cli' => '0000',
'txnid' => '000000403401806110710441878004',
'startdate' => '2018-06-06 14:51:45',
'enddate' => '2018-06-12 00:00:00',
'type' => 'subscription',
'renewalon' => '2018-06-12 00:00:00',
'lastrenewalon' => '2018-06-11 13:06:52',
'fee' => 2.44,
'status' => '0',
'linkid' => '',
)
Now, how can I get the values group by 'mode' from the array value using mysql
You can chain two SUBSTRING_INDEX functions to first get the substring after 'mode', and then get the substring before ,'cli':
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(a, "'mode'", -1), ",'cli'", 1) AS group_condition
Insert the field text in the place of a.
Then you can use group_condition in the GROUP BY clause.

Sum values in relation 3 layers in CakePHP

Huston, I have a problem;
Problem
So, I have a EAV models in my database, Entity, Attribute, Value. The relations between are:
Production hasMany Entity -> Entity belongsTo Production
Entity hasMany Attribute -> Attribute belongsTo Entity
Attribute hasMany Value -> Value belongsTo Attribute
The table values
---value (INT 3) ,
---label (VARCHAR 65),
---id (INT 11 AI)
I Can save the relations in my database, but, my application needs to return the SUM of Value.value a specific production, example:
The final user, need the consolidate of as Production by month of created. So, now, my application return is this:
(int) 0 => array(
'Production' => array(
'myconditions' => 'HereIsNotAProblem'
),
'Entity' => array(
(int) 0 => array(
'id' => '11',
'label' => 'Acompanhamento Familiar',
'production_id' => '8',
'Attribute' => array(
(int) 0 => array(
'id' => '33',
'label' => 'Nascidos vivos no m?s',
'entity_id' => '11',
'Value' => array(
(int) 0 => array(
'id' => '1',
'label' => '< 2500g',
'value' => '0',
'attribute_id' => '33'
),
)
)
)
)
I need to return:
(int) 0 => array(
'Production' => array(
'myconditions' => 'HereIsNotAProblem'
),
'Entity' => array(
(int) 0 => array(
'id' => '11',
'label' => 'Acompanhamento Familiar',
'production_id' => '8',
'Attribute' => array(
(int) 0 => array(
'id' => '33',
'label' => 'Nascidos vivos no m?s',
'entity_id' => '11',
'Value' => array(
(int) 0 => array(
'id' => '1',
'label' => '< 2500g',
'SUM' => 'TOTAL_SUM',
'attribute_id' => '33'
),
)
)
)
)
How I do this?
So when you query your data you need to left join Values to Attributes and use SUM expression. You can see similar example here, but using COUNT: https://book.cakephp.org/3.0/en/orm/query-builder.html#using-leftjoinwith

Typo3 6.2 TCA Backend Form - How to fill Select dynamically in this example

I have 3 Entities:
Appointment
Doctor (extend fe_users)
Specialization
Dependencies:
Each appointment has exactly one doctor
Each doctor has at least one specialization
Each appointment has at least one specialization - but it can't be
one that its doctor doesn't have
My goal: After a doctor is selected you can choose which specialization of the doctor will be used in order to categorize the appointment - but multiselect should also be possible.
Edit: I came up with this sql to make it clearer what I want in my Select-Field:
SELECT s.name
FROM
tx_appointments_domain_model_specialization s,
fe_users fe,
tx_appointments_doctor_specialization_mm ds,
WHERE fe.username = "###REC_FIELD_doctor###"
AND ds.uid_local = fe.uid
AND s.uid = ds.uid_foreign;
So the part of my TCA of Appointment looks like this now:
It's pretty much all generated by the extension builder except the 'foreign_table_where' in doctor which I found out somehow. But I can't wrap my head around how to set the specialization now.
'doctor' => array(
'exclude' => 1,
'label' => 'doctor',
'config' => array(
'type' => 'select',
'foreign_table' => 'fe_users',
'foreign_table_where' => "AND FIND_IN_SET('1', fe_users.usergroup) ORDER BY fe_users.last_name ASC, fe_users.first_name ASC",
'minitems' => 0,
'maxitems' => 1,
),
),
'specialization' => array(
'exclude' => 1,
'label' => 'specialization',
'config' => array(
'type' => 'select',
'foreign_table' => 'tx_appointments_domain_model_specialization',
'MM' => 'tx_appointments_appointment_specialization_mm',
'size' => 10,
'autoSizeMax' => 30,
'maxitems' => 9999,
'multiple' => 0,
'wizards' => array(
'_PADDING' => 1,
'_VERTICAL' => 1,
'edit' => array(
'module' => array(
'name' => 'wizard_edit',
),
'type' => 'popup',
'title' => 'Edit',
'icon' => 'edit2.gif',
'popup_onlyOpenIfSelected' => 1,
'JSopenParams' => 'height=350,width=580,status=0,menubar=0,scrollbars=1',
),
'add' => Array(
'module' => array(
'name' => 'wizard_add',
),
'type' => 'script',
'title' => 'Create new',
'icon' => 'add.gif',
'params' => array(
'table' => 'tx_appointments_domain_model_appointment',
'pid' => '###CURRENT_PID###',
'setValue' => 'prepend'
),
),
),
),
),
I know I could use something like 'foreign_table_where' => ..."###REC_FIELD_doctor###"... but I have no idea how the whole SQL should look like to get only the doctor's specializations showing.
And I'll probably have to add something like this in ext_tables.php which I got from this answer similar to my problem:
$TCA['tx_appointments_domain_model_appointment']['ctrl']['requestUpdate'] .= ',doctor';

How to get multiple records join into one table or use contain with conditions?

I'm using cakePHP 2.6.4.
I have two tables AgentChat and AgentChatMember.
AgentChat hasMany AgentChatMember,
AgentChatMember belongsTo AgentChat.
AgentChat have: id, team_id, agent_chat_member_count fields.
AgentChatMember have: id, agent_chat_id, user_id fields.
I want to find AgentChat by chat_team_id and agent_chat_member_count,
and AgentChatMember as contain with user_id conditions.
$options['contain'] = array('AgentChatMember');
$options['conditions']['AgentChat.agent_chat_member_count'] = count($user_ids);
$options['conditions']['AgentChat.chat_team_id'] = $chat_team_id;
$res = $this->AgentChat->find('first', $options);
and $res is:
'AgentChat' => array(
'id' => '1',
'agent_message_count' => '0',
'agent_chat_member_count' => '2',
'chat_team_id' => '14',
'created' => '2015-05-06 09:52:31',
'updated' => '2015-05-06 09:52:31'
),
'AgentChatMember' => array(
(int) 0 => array(
'id' => '1',
'agent_chat_id' => '1',
'user_id' => '26',
'created' => '2015-05-06 09:52:31'
),
(int) 1 => array(
'id' => '2',
'agent_chat_id' => '1',
'user_id' => '21',
'created' => '2015-05-06 09:52:31'
)
)
By this i get what i want except fact i cant set user_id condition to AgentChatMember like this: $options['conditions']['AgentChatMember.user_id'] = $user_ids;
When i'm using joins instead of contain i can set user_id conditions, but i get result like this:
(int) 0 => array(
'AgentChat' => array(
'id' => '1',
'agent_message_count' => '0',
'agent_chat_member_count' => '2',
'chat_team_id' => '14',
'created' => '2015-05-06 09:52:31',
'updated' => '2015-05-06 09:52:31'
),
'AgentChatMember' => array(
'id' => '2',
'agent_chat_id' => '1',
'user_id' => '21',
'created' => '2015-05-06 09:52:31'
)
),
(int) 1 => array(
'AgentChat' => array(
'id' => '1',
'agent_message_count' => '0',
'agent_chat_member_count' => '2',
'chat_team_id' => '14',
'created' => '2015-05-06 09:52:31',
'updated' => '2015-05-06 09:52:31'
),
'AgentChatMember' => array(
'id' => '2',
'agent_chat_id' => '1',
'user_id' => '26',
'created' => '2015-05-06 09:52:31'
)
),
...
Its possible to get multiple records join into one table or contain with conditions on it?
How can i achieve that?
You can pass conditions to the contain:-
$res = $this->AgentChat->find(
'first',
array(
'contain' => array(
'AgentChatMember' => array(
'conditions' => array(
'AgentChatMember.user_id' => $userIds
)
)
),
'conditions' => array(
'AgentChat.agent_chat_member_count' => count($userIds),
'AgentChat.chat_team_id' => $chatTeamId
)
)
);
Just as an aside, you should really be camel casing your variables in CakePHP. See the Coding Standards in the documentation.

Should the following join structure work for a CakePHP find?

Is the following join structure correctly formatted to work with a CakePHP find? This doesn't seem to be working when I use it along with conditions and my gut tells me that the structure is off. I'm new to joins, so any help is appreciated.
'joins' => array(
(int) 0 => array(
'table' => 'cheeses_milk_sources',
'alias' => 'CheesesMilkSource',
'type' => 'INNER',
'conditions' => array(
(int) 0 => 'Cheese.id = CheesesMilkSource.cheese_id'
)
),
(int) 1 => array(
'table' => 'milk_sources',
'alias' => 'MilkSource',
'type' => 'INNER',
'conditions' => array(
(int) 0 => 'CheesesMilkSource.milk_source_id = MilkSource.id',
'CheesesMilkSource.milk_source_id' => '1'
)
)
),
'conditions' => array(
'AND' => array(
(int) 0 => array(
'Cheese.cheese_producer_id' => (int) 35
),
(int) 1 => array(
'Cheese.active' => (int) 1
)
)
),
The confusing part is
'conditions' => array(
(int) 0 => 'CheesesMilkSource.milk_source_id = MilkSource.id',
'CheesesMilkSource.milk_source_id' => '1'
)
You are mixing an array element without a key with one with a key equal to CheesesMilkSource.milk_source_id.
if you need to specify two conditions on the join, do it like
'conditions' => array(
'CheesesMilkSource.milk_source_id = MilkSource.id AND CheesesMilkSource.milk_source_id = 1'
)
perhaps the next snippet will equally work, but I'm not sure and can't test it at the moment - let me know if it does with a comment.
'conditions' => array(
'CheesesMilkSource.milk_source_id = MilkSource.id',
'CheesesMilkSource.milk_source_id = 1'
)
But since you're joining the tables, maybe you should put CheesesMilkSource.milk_source_id = 1 in the general conditions of the find:
'conditions' => array(
'Cheese.cheese_producer_id' => 35,
'Cheese.active' => 1,
'CheesesMilkSource.milk_source_id' => 1,
)
Notice you don't need to specify AND as this is the default way of joining conditions.