i am trying to solve a performance issue.
I have a table called "pages" which contains thousands of pages.
There is a field called "status" which can be 0,1,2 and i want to select only 1 and 2 but not 0. As soon as i add the additional AND != 0 or IN (1,2) or >= 1 this query takes 0,5 seconds, without it its quiet fast.
Here are my queries:
SELECT name, pages.id, urlString, metaTitle, metaDescription, id,
anchorText
FROM pages
INNER JOIN metaDe
ON pages.id=metaDe.pageId
WHERE id = (SELECT min(id) FROM pages WHERE
linkToPageFunction = ? AND id > ? AND status != 0 )
LIMIT 1
SELECT name, pages.id, urlString, metaTitle,
metaDescription, id, anchorText
FROM pages
INNER JOIN metaDe
ON pages.id=metaDe.pageId
WHERE id = (SELECT max(id) FROM pages WHERE
linkToPageFunction = ? AND id < ? AND status != 0 )
LIMIT 1
without "AND status != 0" it is fast.
I want it to be fast plus if possible i want to use one query which returns two results (previous and next) instead of using two queries.
Any suggestions?
Thanks in advance
EDIT EXPLAIN:
array (
0 =>
array (
'id' => 1,
'select_type' => 'PRIMARY',
'table' => 'metaDe',
'type' => 'const',
'possible_keys' => 'PRIMARY,pageId_2,pageId,pageId_3,idxMetaDePageId',
'key' => 'PRIMARY',
'key_len' => '4',
'ref' => 'const',
'rows' => 1,
'Extra' => 'Using where',
),
1 =>
array (
'id' => 1,
'select_type' => 'PRIMARY',
'table' => 'pages',
'type' => 'const',
'possible_keys' => 'PRIMARY,id,id_2,id_3',
'key' => 'PRIMARY',
'key_len' => '4',
'ref' => 'const',
'rows' => 1,
'Extra' => 'Using where; Using index',
),
2 =>
array (
'id' => 2,
'select_type' => 'SUBQUERY',
'table' => 'pages',
'type' => 'ref',
'possible_keys' => 'PRIMARY,linkToPageFunction,id,status,id_2,id_3,idxPagesForStatusQuery,idxPagesStatus',
'key' => 'idxPagesForStatusQuery',
'key_len' => '93',
'ref' => 'const',
'rows' => 298725,
'Extra' => 'Using where; Using index',
),
)array (
0 =>
array (
'id' => 1,
'select_type' => 'PRIMARY',
'table' => 'pages',
'type' => 'const',
'possible_keys' => 'PRIMARY,id,id_2,id_3',
'key' => 'PRIMARY',
'key_len' => '4',
'ref' => 'const',
'rows' => 1,
'Extra' => 'Using index',
),
1 =>
array (
'id' => 1,
'select_type' => 'PRIMARY',
'table' => 'metaDe',
'type' => 'const',
'possible_keys' => 'PRIMARY,pageId_2,pageId,pageId_3,idxMetaDePageId',
'key' => 'PRIMARY',
'key_len' => '4',
'ref' => 'const',
'rows' => 1,
'Extra' => '',
),
2 =>
array (
'id' => 2,
'select_type' => 'SUBQUERY',
'table' => 'pages',
'type' => 'range',
'possible_keys' => 'PRIMARY,linkToPageFunction,id,status,id_2,id_3,idxPagesForStatusQuery,idxPagesStatus',
'key' => 'idxPagesForStatusQuery',
'key_len' => '97',
'ref' => NULL,
'rows' => 5,
'Extra' => 'Using where; Using index',
),
)
Since i cannot answer this thread for whatever reason, i am giving my answer and solutions here.
Thanks everybody. Sometimes you need to thing and talk about to fix it. I guess i just found the (simple) solution.
Thats what i am doing now. I have no idea why i havent got this way before.
Next:
SELECT name, pages.id, urlString, metaTitle, metaDescription, id, anchorText
FROM pages
INNER JOIN metaDe
ON pages.id=metaDe.pageId
WHERE id > ? AND status IN (1, 2) AND linkToPageFunction = ?
ORDER BY id ASC LIMIT 1
Previous:
SELECT name, pages.id, urlString, metaTitle, metaDescription, id, anchorText
FROM pages
INNER JOIN metaDe
ON pages.id=metaDe.pageId
WHERE id < ? AND status IN (1, 2) AND linkToPageFunction = ?
ORDER BY id DESC LIMIT 1
You dont seem to have an index on your pages table that includes the status field. Add such an index and your query speed should improve a lot.
create index idxPagesStatus on pages (status);
or even
create index idxPagesForStatusQuery on pages (linkToPageFunction, id, status);
to include all fields in the query. Also, on your metaDe tabke pageId could use an index as well.
create index idxMetaDePageId on metaDe (pageId);
also, if you know the pages.id key is sequential for sure(ie, goes like: 1, 2, 3, 4, 5,...), and none is deleted, you could simply do:
SELECT
name,
pages.id,
urlString,
metaTitle,
metaDescription,
id,
anchorText
FROM
pages
left JOIN
metaDe ON pages.id=metaDe.pageId
WHERE
abs(2-id)=1;
I want to create a table with a LONGBLOB field in it.
'image' => $this->binary(),
produces BLOB field in the table.
Is there any other way to produce LONGBLOB field except using raw SQL syntax for the specific field.
Below is my full code for creating the table.
$this->createTable('edition_images', [
'image_id' => $this->bigPrimaryKey()->unsigned(),
'embed_url' => $this->string()->notNull(),
'image_type' => $this->string(),
'image_md5' => $this->string(),
//'image' => $this->binary(),
'`image` longblob NULL',
'title_en' => $this->string(),
'title_bg' => $this->string(),
'title_ro' => $this->string(),
'order' => $this->bigInteger(20)->unsigned()->null(),
'edition_id' => $this->bigInteger(20)->unsigned()->notNull(),
'created_by' => $this->bigInteger(20)->unsigned()->notNull(),
'created_at' => $this->timestamp()->notNull()->defaultExpression('CURRENT_TIMESTAMP'),
'updated_by' => $this->bigInteger(20)->unsigned()->null(),
'updated_at' => $this->timestamp()->null()->defaultValue(null)->append('ON UPDATE CURRENT_TIMESTAMP'),
'deleted_by' => $this->bigInteger(20)->unsigned()->null(),
'deleted_at' => $this->timestamp()->null(),
'deleted' => $this->integer(1),
]);
You can pass the exact column type as text:
'image' => 'LONGBLOB'
You should be able to specify the length as a parameter in the binary method. Since the longblob is 4GB you have to specify this in bytes:
'image' => $this->binary(4294967295),
However, $length is being ignored. The code
$this->db->createCommand()->createTable("test_blob", [
"id" => $this->integer(),
"datum" => $this->binary(429496729),
"txt" => $this->string()
])->getSql();
returns the following SQL:
CREATE TABLE `test_blob` (
`id` int(11),
`datum` blob,
`txt` varchar(255)
);
I've added an issue on Github
I have multiple array to insert into database but i don't fix the field name because can select format table data and insert into database but can check field name with $id_template.
This my format table(example)
So i want to know how can i get data from multiple array to insert into database
This my code in controller
$column = $this->m_rate_template->get_column($id_template);
$colum_detail = implode(",", $column);
$column_cut = explode(",", $colum_detail); //example data get format is Array ( [0] => min [1] => max)
foreach ($column_cut as $key => $val){
$a = $this->input->post($column_cut[$key]);
foreach ($a as $key1 => $val1){
echo $val1;
$child_data = array(
'id' => $this->m_rate_template->generate_id_in_template($template_name),
'id_rate' => $id_rate,
$column_cut[$key] => $val1
);
$this->m_rate_template->insert_rate($child_data, $template_name);
}
}
My data it show like this
Array ( [id] => 4ae665037e [id_rate] => 7f881e02bb [min] => 1 )
Array ( [id] => bc3e60157f [id_rate] => 7f881e02bb [min] => 2 )
Array ( [id] => 082de3ad82 [id_rate] => 7f881e02bb [max] => 1 )
Array ( [id] => ee135ecd8a [id_rate] => 7f881e02bb [max] => 2 )
actually, data should be like this
Array ( [id] => 4ae665037e [id_rate] => 7f881e02bb [min] => 1 [max] => 2)
Array ( [id] => 082de3ad82 [id_rate] => 7f881e02bb [max] => 1 [max] => 2)
Update
$array = array(
[0] => array(
'min' => '2500',
'max' => '5000'
),
[1] => array(
'min' => '5001',
'max' => '7000'
)
)
You can use batch insert to insert multiple
$this->db->insert_batch();
first parameter is table name and second is array of arrays(records)
if you have want to insert multiple record in table then you can also use codeigniter inbuilt insert_batch function without make query in loop.
so i thing your execution will be fast.
you have want to array in below format.
$array = array(
[0] => array(
'column 1' => 'value 1',
'column 2' => 'value 1'
),
[1] => array(
'column 1' => 'value 2',
'column 2' => 'value 2'
)
)
$this->db->insert_batch('tbl_name',$array)
so please make your code and generate your array as above in loop and simply pass your array in insert_batch function.
I have the following query in a Cakephp 2.4 model:
$scores = $this->WorkInfo->find('list', array(
'conditions' => array('WorkInfo.work_id' => $work_ids),
'fields' => array('WorkInfo.date', 'SUM(WorkInfo.score)'),
'group' => array('WorkInfo.date')
));
Which generates the following query:
SELECT
`WorkInfo`.`date`,
SUM(`WorkInfo`.`score`)
FROM
`home`.`work_infos` AS `WorkInfo`
WHERE
`WorkInfo`.`work_id` IN (4, 7, 8, 12, 9, 11, 13, 10, 14, 6, 5)
GROUP BY
`WorkInfo`.`date`
The result I get in my application is:
'2014-03-24' => null
'2014-03-25' => null
'2014-03-26' => null
'2014-03-27' => null
'2014-03-28' => null
'2014-03-31' => null
While the result I get from pasting this very query in the mysql console is:
'2014-03-24' => 0
'2014-03-25' => 36
'2014-03-26' => 0
'2014-03-27' => 164
'2014-03-28' => 0
'2014-03-31' => 0
What is going on here? It is supposed that same queries output same results, isn't it?
I have read something about creating virtual fields for this, but I do not want to overkill, it should be possible to perform a simple aggregation query through Cakephp using the find function.
Thanks!
Try this
$scores = $this->WorkInfo->find('all', array(
'conditions' => array('work_id' => $work_ids),
'fields' => array('date', 'SUM(score) AS score'),
'group' => array('date')
));
then with Set::combine you can format your array cakephp find list
$scores = Set::combine($scores, '{n}.WorkInfo.date', '{n}.0.score');
prints=>
'2014-03-24' => 0
'2014-03-25' => 36
'2014-03-26' => 0
'2014-03-27' => 164
'2014-03-28' => 0
'2014-03-31' => 0
Ok, sadly, I think what you want to do can't be done as you want to do it.
Let's see, you use the find('list') method, so that's here in the API. Code looks normal, and as you said, query is ok, returns everything you want. Problem is in line 2883
return Hash::combine($results, $query['list']['keyPath'], $query['list']['valuePath'], $query['list']['groupPath']);
That line organizes the returned array after the query is done. And seeing the doc for that function, we have
Creates an associative array using a $keyPath as the path to build its
keys, and optionally $valuePath as path to get the values. If
$valuePath is not specified, or doesn’t match anything, values will be
initialized to null.
Which is what happens to you. Now, debugging, the query result before applying the Hash::combine function is something like this
Array
(
[0] => Array
(
[WorkInfo] => Array
(
[date] => 2013-04-01
)
[0] => Array
(
[SUM(`WorkInfo`.`score`)] => 24
)
)
)
so you see, you get the results. And the respective Hash::combine
Array
(
[groupPath] =>
[valuePath] => {n}.SUM(WorkInfo.score)
[keyPath] => {n}.WorkInfo.date
)
which probably causes problem with the dot inside the parenthesis. And the combine function doesn't find the valuePath, and you get null, and you get sad.
If you change your query to 'SUM(WorkInfo.score) AS score' (leaving everything as is), you have almost the same problem with valuePath
Array
(
[groupPath] =>
[valuePath] => {n}.SUM(WorkInfo.score) as score
[keyPath] => {n}.WorkInfo.date
)
//respective result array
Array
(
[0] => Array
(
[WorkInfo] => Array
(
[date] => 2013-04-01
)
[0] => Array
(
[score] => 24
)
)
)
You might think that doing 'SUM(score) AS score' (without the dot) will solve it, but the code of find('list') adds the alias if it doesn't find a dot (in line 2865).
So... I guess what I'm saying is: do a virtual field, or listen to Isaac Rajaei, or create a custom find function. But with find('list') and SUM() you won't have luck :(
I need to build WP_Query that will return data from database where both conditions are fulfilled, because of that I'm using AND as relation but returned data are different than I expected. To make it more clear I will post WP_Query arguments here.
Array(
[post_type] => Array
(
[0] => event
)
[post_status] => publish
[paged] => 1
[posts_per_page] => 1000
[tax_query] => Array
(
[relation] => AND
[1] => Array
(
[taxonomy] => event_dates
[field] => slug
[terms] => Array
(
[0] => thursday
[1] => exhibitions
)
)
)
)
With this arguments I thought that I'll get events with type exibition AND are on Thursday.
Thanks in advance for help.
The AND is on the wrong level. The WP_Query arguments should look like something like this:
...
'tax_query' => array(
array(
'taxonomy' => 'event_dates',
'field' => 'slug',
'terms' => array('thursday', 'exhibitions'),
'operator' => 'AND'
)
)
...