I have a database table of sales that has 100k records. In the future months the size will double and later triple and etc.. The table has relation with two more tables that I need data from. Finally I want to list everything in GridView with ActiveDataProvider but it takes waaay to long(3-5min). I am wondering what is the best way dealing with this situation?
My current query looks like this:
$query = Sale::find()->select([
's.*',
'u.username as userName',
'ship.name as ShipName'
])
->joinWith('user')
->joinWith('shipping')
$dataProvider = new ActiveDataProvider([
'query' => $query,
'pagination' => [
'defaultPageSize' => 25,
'pageSizeLimit' => 1000],
]);
As you can see its a simple query, but the data is too huge to work normally.. What can I do?
Related
please help I dont know how to get relation table when using sqldataprovider. Anyone understand how to use relation model?
$model = new Finalresult();
$searchModel = new FinalresultSearch();
$dataProvider = $searchModel->search(Yii::$app->request->queryParams);
$dataProvider = new SqlDataProvider([
'sql' => 'SELECT finalresult.bib,
finalresult.series_id,
finalresult.category_id,
GROUP_CONCAT(DISTINCT finalresult.point ORDER BY series.serie_seri_no DESC) AS seriPoint
FROM finalresult, series GROUP BY finalresult.bib',
'key' => 'bib',
]);
I'm trying to get relation table:
'attribute'=>'category_id',
'width'=>'300px',
'value'=>function ($model, $key, $index, $widget) {
return $model->category->category_name;
},
then getting trying to non-object
You can't use relations with SqlDataProvider, because each single result will be presented as array, for example:
[
'id' => 1,
'name' => 'Some name',
'category_id' => 1,
],
For example, you can access category_id as `$model['category_id'].
SqlDataProvider is for very very complex queries, your query can easily be written as ActiveQuery and you can use ActiveDataProvider and get all advantages of that (relations, etc.).
You can find category by id, but it will be lazily loaded that means amount of queries is multiplied by number of rows.
With ActiveDataProvider and relations you can use eager loading and reduce amount of queries. Read more in official docs.
Grid Columns example in documentation
try to change "value" to
'value'=> function($data) {
return $data['category']['category_name'];
}
I'm trying to create an AJAX form whereby the content of a select field populates based on the choice of a preceding select field (you see this a lot with 'country' populating 'state/province'). In my case, I want users to be able to choose their province only if active accounts exist in it.
The Javascript I can write no problem. Fetching the data is where I'm... not so much stuck as doing too much work. CakePHP likes to build select fields with options in an array of the form
$options = array(select_option_value => display_text)
My strategy, though functional, must be more convoluted than cake intended (this a is segment of a controller method).
$provinceData = $this->Account->find('all',array('recursive' => 0,
'joins' => array(
array(
'table' => 'provinces',
'type' => 'LEFT',
'conditions' => array('Account.province_id = provinces.id')
)),
'fields'=>array('provinces.id', 'provinces.name', 'provinces.abbrev'),
'conditions' => array('registration > 2')));
$provinces = array();
foreach($provinceData as $pd) {
/*note: lowercase, plural below b/c can't get 'alias' => 'Province'
to work in joins array above : ( */
$id = $pd['provinces']['id'];
$name = $pd['provinces']['name'];
$provinces[$id] = $name;
}
$this->set(compact('provinces'));
Can anyone point out a more appropriate way to do this? I assume there must be a MySQL query that can do this, but I'm pretty bad at writing elaborate MySQL queries in the first place, let alone via Cake's convention (and, for you MySQL gurus out there, I'm happy to do this from a Model->query(//MySQL code) call instead!
Any and all help truly appreciated.
Assuming the relationship Account belongsTo Province you can try this code:
$accounts = $this->Account->find(
'all',
array(
'fields' => array('Account.province_id', 'Province.name'),
'conditions' => array('Account.registration > 2'),
'group' => 'Account.province_id'
)
);
$provinces = Hash::combine($accounts, '{n}.Account.province_id', '{n}.Province.name');
$this->set(compact('provinces'));
edit: missed bracket and a period instead of an underscore . Now should work
I am not very good with DB queries. And with Yii it's more complicated, since I am not very used to it.
I need to optimize a simple query
$userCalendar = UserCalendar::model()->findByAttributes(array('user_id'=>$user->id));
$unplannedEvents = CalendarEvent::model()->findAllByAttributes(array('calendar_id'=> $userCalendar->calendar_id,'planned'=>0));
CalendarEvent table, i.e the second table from which I need records does not have an user_id but a calendar_id from which I could get user_id from UserCalendar, i.e. the first table hence I created a UserCalendar object which is not a very good way as far as I understand.
Q1. What could I do to make it into one.
Q2. Yii does this all internally but I want to know what query it built to try it seperately in MySQL(phpMyAdmin), is there a way to do that?
Thanks.
Q1: You need to have the relation between UserCalendar and CalendarEvent defined in both of your active record models (in the method "relations").
Based on your comments, it seems like you have the Calendar model that has CalendarEvent models and UserCalendar models.
Lets assume your relations in Calendar are:
relations() {
return array(
'userCalendar' => array(self::HAS_MANY, 'UserCalendar', 'calendar_id'),
'calendarEvent' => array(self::HAS_MANY, 'CalendarEvent', 'calendar_id'),
}
In CalendarEvent:
relations() {
return array( 'calendar' => array(self::BELONGS_TO, 'Calendar', 'calendar_id'), );
}
And in UserCalendar:
relations() {
return array( 'calendar' => array(self::BELONGS_TO, 'Calendar', 'calendar_id'), );
}
So to make the link between UserCalendar and CalendarEvent you'll need to use Calendar
$criteria = new CDbCriteria;
$criteria->with = array(
"calendarEvent"=>array('condition'=>'planned = 0'),
"userCalendar"=>array('condition'=> 'user_id =' . $user->id),
);
$calendar = Calendar::model()->find($criteria);
and $calendar->calendarEvent will return an array of calendarEvent belonging to the user
Q2: you can enable web logging so all the db request (and others stuffs) will appear at the end of your page:
Logging in Yii (see CWebLogging)
In your application configuration put
'components'=>array(
......
'log'=>array(
'class'=>'CLogRouter',
'routes'=>array(
array(
'class'=>'CWebLogRoute',
),
),
),
),
I really don't know enough about MySQL queries and it's showing.
I have a custom field set for every post. The custom field stores the posts source URL in a key called "source_url".
I have it working with the below WP_Query parameters, but it's incredibly slow. Keep in mind it's possible to 50+ urls to search for.
So, given an array of source URL's, I want to fetch the matching posts.
For example, here is what I currently have that's slow in WP_Query:
// var_dump of $urls array (this could be 50+ urls)
array(7) {
[0]=>
string(42) "http://www.youtube.com/watch?v=FMghvnqDhT8"
[1]=>
string(42) "http://www.youtube.com/watch?v=RY-yUFpXTnM"
[2]=>
string(58) "http://www.youtube.com/watch?v=nIm2dnyJ1Ps&feature=related"
[3]=>
string(42) "http://www.youtube.com/watch?v=NoCtRQlJAqM"
[4]=>
string(57) "http://holidaycustoms.blogspot.com/2012/08/busy-week.html"
[5]=>
string(42) "http://www.youtube.com/watch?v=DcZvg197Ie4"
[6]=>
string(42) "http://www.youtube.com/watch?v=7P3UEbLmLuo"
}
// Build Media Query
$meta_query = array(
'relation' => 'OR'
);
foreach( $urls as $url ) {
$meta_query[] = array(
'key' => 'source_url',
'value' => $url
);
}
// Get 20 matching posts from a category set by a variable
$args = array(
'post_type' => 'post',
'posts_per_page' => 20,
'orderby' => 'rand',
'cat' => $cat_ID,
'meta_query' => $meta_query
);
$posts = get_posts($args);
What I'm looking to do is replace the above code with a custom query select, which I have read is much faster than WP_Query.
But I don't know enough about MySQL or the WP database to build the custom select query. Can anyone help? Thanks in advance!
In the post you linked yourself, the first reply already states that
[...] the default schema doesn't even have an index on the value column
Which is far more severe a problem than any you would have with a query generator, because without an index the DBMS has to traverse the whole table and compare strings of each field.
Adding an index is fairly easy with an appropriate management tool like PHPMyAdmin. The offending table you will need to add an index to is called wp_postmeta and the field that needs an index is meta_value, and the index type should be INDEX.
Adding an index is transparent and does not affect wordpress other than in performance. It could take some time though since, well MySQL needs to traverse the whole table. Also, because you are indexing string data, the index will be quite big.
You should also try using appropriate structures for your query. You are currently using a big ORed selection with different values but always the same field. There is a construct for just that, and it's called IN.
...
// Build Media Query
$meta_query = array();
$meta_query[] = array(
'key' => 'source_url',
'value' => $urls,
'compare' => 'IN'
);
// Get 20 matching posts from a category set by a variable
..
(Untested. I actually never did this, Reference)
The performance gain would be negligible compared to adding an index I assume, but your code would become a lot simpler.
I have a News model, which has a HABTM relationship with an Artists model, and an artist in turn hasMany tourdates.
If I want to find all tourdates related to the current news item, what is an efficient way of phrasing that for CakePHP?
This is what I have so far; I'm wondering if (a) it looks like it should work, and (b) if there's any more concise way of writing it:
$relatedartists = $this->News->ArtistsNews->find('list', array(
'conditions'=>array('ArtistsNews.news_id' => $id),
'fields'=>array('artist_id')
));
$livedates = $this->News->Artists->Tour->find('all', array(
'conditions'=>array('Tour.artist_id'=> $relatedartists,
'date >= ' . time()),
'order'=>'date ASC'
));
What you have is pretty good. I always prefer to use multiple queries rather than use massive joins which create temporary tables. It can reduce performance somewhat.
You might also try something like the below
$opts = array(
'conditions' => array(
'ArtistsNews.news_id' => $id
)
);
$this->News->Artists->recursive = 2;
$this->News->Artists->find('all', $opts);
Something along the likes of this query will also get you what you need (haven't error checked)