we use elasticsearch to power our classified search engine Listings360 Kenya.
In Elasticsearch version 6 during the search we could get the total hits documents count used for pagination here from this json output
Array
(
[took] => 3
[timed_out] =>
[_shards] => Array
(
[total] => 5
[successful] => 5
[skipped] => 0
[failed] => 0
)
[hits] => Array
(
[total] => 30540
[max_score] =>
[hits] => Array
(
[0] => Array
(
Now the exact same search with elasticsearch 7 give the following json output
Array
(
[took] => 14
[timed_out] =>
[_shards] => Array
(
[total] => 1
[successful] => 1
[skipped] => 0
[failed] => 0
)
[hits] => Array
(
[total] => Array
(
[value] => 10000
[relation] => gte
)
[max_score] =>
[hits] => Array
(
[0] => Array
(
You can see that i don't have the [hits][total] anymore which is used for pagination purpose.
Any idea how to get that back
Thank you for your help
jap, that´s one of the breaking changes in 7.0.
Have a look at the search param track_total_hits which forces the count to always be accurate. Nevertheless, the new format will still apply and you need to patch your application (that's why it's a breaking change).
Here is more detailed info regarding the new response structure.
I have the following query in a Cakephp 2.4 model:
$scores = $this->WorkInfo->find('list', array(
'conditions' => array('WorkInfo.work_id' => $work_ids),
'fields' => array('WorkInfo.date', 'SUM(WorkInfo.score)'),
'group' => array('WorkInfo.date')
));
Which generates the following query:
SELECT
`WorkInfo`.`date`,
SUM(`WorkInfo`.`score`)
FROM
`home`.`work_infos` AS `WorkInfo`
WHERE
`WorkInfo`.`work_id` IN (4, 7, 8, 12, 9, 11, 13, 10, 14, 6, 5)
GROUP BY
`WorkInfo`.`date`
The result I get in my application is:
'2014-03-24' => null
'2014-03-25' => null
'2014-03-26' => null
'2014-03-27' => null
'2014-03-28' => null
'2014-03-31' => null
While the result I get from pasting this very query in the mysql console is:
'2014-03-24' => 0
'2014-03-25' => 36
'2014-03-26' => 0
'2014-03-27' => 164
'2014-03-28' => 0
'2014-03-31' => 0
What is going on here? It is supposed that same queries output same results, isn't it?
I have read something about creating virtual fields for this, but I do not want to overkill, it should be possible to perform a simple aggregation query through Cakephp using the find function.
Thanks!
Try this
$scores = $this->WorkInfo->find('all', array(
'conditions' => array('work_id' => $work_ids),
'fields' => array('date', 'SUM(score) AS score'),
'group' => array('date')
));
then with Set::combine you can format your array cakephp find list
$scores = Set::combine($scores, '{n}.WorkInfo.date', '{n}.0.score');
prints=>
'2014-03-24' => 0
'2014-03-25' => 36
'2014-03-26' => 0
'2014-03-27' => 164
'2014-03-28' => 0
'2014-03-31' => 0
Ok, sadly, I think what you want to do can't be done as you want to do it.
Let's see, you use the find('list') method, so that's here in the API. Code looks normal, and as you said, query is ok, returns everything you want. Problem is in line 2883
return Hash::combine($results, $query['list']['keyPath'], $query['list']['valuePath'], $query['list']['groupPath']);
That line organizes the returned array after the query is done. And seeing the doc for that function, we have
Creates an associative array using a $keyPath as the path to build its
keys, and optionally $valuePath as path to get the values. If
$valuePath is not specified, or doesn’t match anything, values will be
initialized to null.
Which is what happens to you. Now, debugging, the query result before applying the Hash::combine function is something like this
Array
(
[0] => Array
(
[WorkInfo] => Array
(
[date] => 2013-04-01
)
[0] => Array
(
[SUM(`WorkInfo`.`score`)] => 24
)
)
)
so you see, you get the results. And the respective Hash::combine
Array
(
[groupPath] =>
[valuePath] => {n}.SUM(WorkInfo.score)
[keyPath] => {n}.WorkInfo.date
)
which probably causes problem with the dot inside the parenthesis. And the combine function doesn't find the valuePath, and you get null, and you get sad.
If you change your query to 'SUM(WorkInfo.score) AS score' (leaving everything as is), you have almost the same problem with valuePath
Array
(
[groupPath] =>
[valuePath] => {n}.SUM(WorkInfo.score) as score
[keyPath] => {n}.WorkInfo.date
)
//respective result array
Array
(
[0] => Array
(
[WorkInfo] => Array
(
[date] => 2013-04-01
)
[0] => Array
(
[score] => 24
)
)
)
You might think that doing 'SUM(score) AS score' (without the dot) will solve it, but the code of find('list') adds the alias if it doesn't find a dot (in line 2865).
So... I guess what I'm saying is: do a virtual field, or listen to Isaac Rajaei, or create a custom find function. But with find('list') and SUM() you won't have luck :(
At the moment "matrix_mct_versions" is a table with 73 entries. When I run this query the "version_count" always returns 73, ie the full number of rows. When I run the sub select query on its own i get the real count as per the com_ID param sent. I cannot see what I am doing wrong with this.. can anyone help?
SELECT
a_ID as com_ID,
option_number,
comment,
word_count,
gender,
sample,
(
SELECT
count(a_ID)
FROM
matrix_mct_versions
WHERE
com_ID = com_ID
) as version_count
FROM
matrix_mct
WHERE
attribute_number = :attribute_number AND
grade_number = :grade_number AND
attribute_type = :attribute_type
ORDER BY
option_number
Returns results like this:
[0] => Array
(
[com_ID] => 678
[option_number] => 1
[comment] => TODO primary function missing for controller y
[word_count] => 7
[gender] => 2
[sample] => 0
[version_count] => 73
)
[1] => Array
(
[com_ID] => 679
[option_number] => 2
[comment] => TODO make this green
[word_count] => 4
[gender] => 2
[sample] => 0
[version_count] => 73
)
[2] => Array
(
[com_ID] => 680
[option_number] => 3
[comment] => TODO make this better
[word_count] => 4
[gender] => 2
[sample] => 0
[version_count] => 73
)
At least one problem is your subquery. It is not correlated. I think you mean:
(SELECT count(a_ID)
FROM matrix_mct_versions
WHERE matrix_mct_versions.com_ID = matrix_mct.com_ID
) as version_count
$categ = $this->FreeadsCategory->bindModel( array( 'hasMany' => array( 'Subcategory' => array('foreignKey' => 'category_id', 'order'=>'id ASC') ) ) );
$data = $this->FreeadsCategory->findById($i);
$this->set("datas", $data);
I am not able to fetch the datas in view page using cakephp
If i give pr($datas); showing nothing in ctp file
If i print the data in controller i am getting the following array structure
Array
(
[FreeadsCategory] => Array
(
[id] => 1
[uuid] => 51512434-e4c4-441b-b90e-16f8732d5573
[category] => Automobiles
[status] => Active
)
[Subcategory] => Array
(
[0] => Array
(
[id] => 1
[uuid] => 4ea15f22-adf0-4020-b35d1-052ff9ff9a27
[category_id] => 1
[subcategory] => Cars/Cabs/Jeeps
[status] => Active
)
[1] => Array
(
[id] => 5
[uuid] => 51cec363-e7ac-4095-a86b-0ccdf260d1b4
[category_id] => 1
[subcategory] => Buses/Lorries
[status] => Active
)
)
You don't fetch data in views, that violates the MVC pattern. Technically there are ways to do it but it's plain wrong, you'll end up with unmaintanable garbage code.
I really recommend you to get started by reading about the MVC design pattern and to do the CakePHP blog tutorial first to get a minimum of understanding of how CakePHP works.
The MySQL query I'm currently trying to perform is functionally equivalent to this:
SELECT small_table.A, small_table.B, small_table.C, huge_table.X, huge_table.Y
FROM small_table LEFT JOIN huge_table
ON small_table.D = huge_table.Z
WHERE small_table.E = 'blah'
except that the query doesn't appear to terminate (at least not within a reasonable amount of time), probably because the second table is huge (i.e. 7500 rows with a total size of 3 MB). Can I perform a functionally equivalent join in a reasonable amount of time, or do I need to introduce redundancy by adding columns from the huge table into the small table. (I'm a total beginner to SQL.)
The clause WHERE small_table.E = 'blah' is static and 'blah' never changes.
Here is the EXPLAIN output as requested:
Array ( [0] => Array ( [0] => 1 [id] => 1 [1] => SIMPLE [select_type] => SIMPLE [2] => small_table [table] => small_table [3] => ref [type] => ref [4] => E [possible_keys] => E [5] => E [key] => E [6] => 1 [key_len] => 1 [7] => const [ref] => const [8] => 1064 [rows] => 1064 [9] => Using where [Extra] => Using where ) [1] => Array ( [0] => 1 [id] => 1 [1] => SIMPLE [select_type] => SIMPLE [2] => huge_table [table] => huge_table [3] => eq_ref [type] => eq_ref [4] => PRIMARY [possible_keys] => PRIMARY [5] => PRIMARY [key] => PRIMARY [6] => 4 [key_len] => 4 [7] => my_database.small_table.D [ref] => my_database.small_table.D [8] => 1 [rows] => 1 [9] => [Extra] => ) )
A few things ...
1) Are you executing this query directly in MySQL (either Workbench GUI or command line), or is this query embedded in PHP code? Your EXPLAIN output seems to suggest PHP. If you haven't done so already, try executing the query directly in MySQL and take PHP out of the mix.
2) Your EXPLAIN output looks Ok, except I'm wondering about your WHERE clause with small_table.E = 'blah'. The EXPLAIN output shows that there's an index on column E but the key length = 1, which is not consistent to the comparison with 'blah'. What data type did you use for the column definition for small_table.E?
3) MySQL is estimating that it needs to scan 1064 rows in small_table. How many total rows are in small_table, and how many do you expect should match this particular query?