I have a huge list of IDs that i need to query through a table to find if those IDs are available in the table, if yes fetch its model.
Since there are few thousands of IDs this process is really slow as I'm using CActiveRecord::find() mothod
ex. $book = Book::model()->find('book_id=:book_id', array(':book_id'=>$product->book_id));
I even indexed all possible keys, still no improvement.
Any suggestions to improve the execution speed?
thanks in advance :)
1)
Make a list of book ids
foreach $product in Product-List
$book_ids[$product->book_id] = $product->book_id;
Now query all Book models ( indexed by book_id )
$books = Book::model()->findAll(array(
'index' => 'book_id',
'condition' => 'book_id IN (' . implode(',', $book_ids). ')',
));
Integrate $books in your code, I believe you are looping through all products.
foreach $product in Product-List
if( isset($books[$product->book_id]) )
$model = $books[$product->book_id]
2) Another way (I am just assuming you have Product model)
in Product model add a relation to Book
public function relations() {
.......
'book'=>array(self::HAS_ONE, 'Book', 'book_id'),
.......
}
While retrieving your product list, add 'with' => array('book') condition, with any of CActiveDataProvider or CActiveRecord ...
//Example
$productList = Product::model()->findAll(array(
'with' => array('book'),
));
foreach( $productList as $product ) {
.......
if( $product->book != null )
$model = $product->book;
......
}
with either way you can reduce SQL queries.
Better if you use schema caching because Yii fetches schema each time we execute a query. It will improve your query performance.
You can enable schema caching by doing some configuration in config/main.php file.
return array(
......
'components'=>array(
......
'cache'=>array(
'class'=>'system.caching.CApcCache', // caching type APC cache
),
'db'=>array(
...........
'schemaCachingDuration'=>3600, // life time of schema caching
),
),
);
One more thing you can fetch specific column of the table that will improve performance also.
You can do it by using CDbCriteria with find method of CActiveRecord.
$criteria = new CDbCriteria;
$criteria->select = 'book_id';
$criteria->condition = 'book_id=:book_id';
$criteria->params = array(':book_id'=>$product->book_id);
$book = Book::model()->find($criteria);
I would suggest you to use any nosql database if you are processing thousands of records if that is suitable.
Related
Is there anyone who knows how to do this without the technique of doing it in a one query string. I mean the popular ways I see on the net is by looping in data(the updates) and generating a single update statement and then fire a query. Is it possible for an Eloquent Approach or DB without looping?
This is posible with Eloquent, it might be necessary to enable mass-assignment, but you will get an error if so.
$post_data = Input::all();
$model = Model::find($id);
$model ->fill($post_data);
$model ->save();
or
$post_data = Input::all();
Model::find($id)->update($post_data);
Yes, you can do that but in that case, you have to make the array of data that is a loop is needed to store the data in the array with respective field_name => value of the table.
The following is the example:
$Array = array(); //This is needed to hold data while looping over $YourData
$YourData - is the array of data you want to store in the respective table.
foreach ($YourData as $YourDatakey => $YourDatavalue ){
$Array = [
'table_column_name' => $YourDatavalue['value_from_array'],
'table_column_name' => $YourDatavalue['value_from_array'],
'table_column_name' => $YourDatavalue['value_from_array'],
...... and so on
];
}
$InsertQuery= YourModelName::create($Array);
PS:
YourModelName model file should have the columns in protected
$fillable = ['column1','column2'....];
You should use App\Models\ModelName; at the top of the file.
I'm trying to get a subset of results as a virtualField for use in my view. I may even be way off on how I'm approaching this, but here's what I've done so far:
I started with this question here: CakePHP virtualField find all not null which lead to this little beauty.
Now I have an issue where the find statement passing (Array) into the MySQL.
My code looks like:
class Transaction extends AppModel {
public function __construct($id = false, $table = null, $ds = null) {
parent::__construct($id, $table, $ds);
$this->virtualFields['Accounts'] = $this->find("all", array("conditions" => array("account !=" => null)));
}
And I'm seeing:
Error: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'Array' in 'field list'
SQL Query: SELECT `Transaction`.`id`, `Transaction`.`name`,
`Transaction`.`person_id`, `Transaction`.`account`, (Array)
AS `Transaction__Accounts` FROM `my_database`.`transactions`
AS `Transaction` WHERE `Transaction`.`person_id` = (2)
I've also tried $this->Transaction->find and "Transaction.account !=", to no avail. I've found some other issues with the (Array) but none that help my situation. Any pointers in the right direction would be great.
Problem: your query results are an array, and you're telling SQL to assign a field name to each query result containing that array - virtual fields are only made to contain single level variables like strings.
Solution: use a join structure onto itself with those conditions which will return a nested result set along with each of your results. Use CakePHP's model relationships to do this:
<?php
class Transaction extends AppModel {
var $hasMany = array(
'Accounts' => array(
'className' => 'Transaction',
'foreignKey' => false,
'conditions' => array('Accounts.account IS NOT NULL')
)
);
}
?>
Example output:
Array(
'Transaction' => array( // transaction data),
'Accounts' => array( // associated transaction data with account set to null
)
Now, as you can probably gather from that result, if you return 1000 rows from Transaction, you'll get all results from Accounts nested into each Transaction result. This is far from ideal. From here, you can either make the join conditions more specific to target relevant Accounts records, or this is not the right approach for you.
Other approaches could be:
Accounts model, uses Transaction database table, implicit find conditions are that account is null
Manual query to retrieve these results in the afterFind() method of your Transaction model, which will retrieve these results once, and you'll then return array_merge($accounts, $transactions)
I'm not savvy with MySQL or databases generally, so here's a model of my data (table{cols}]) in order to make my question coherent:
Domains{id, name} Note: 'domains' here does not refer to web domains
Subdomains{id, domain_id, name}
Items{id, subdomain_id, name}
SubdomainsItems{id, subdomain_id, item_id} no domain_id column!
My Items Controller has a function, fetchWithin($domains, $subdomains) which, ultimately, should just execute one of two complexish find(). It's the complexish I can't get past.
Programmatically I can achieve this, but I'm quite certain the better way is by clever joins and the like. Alas, currently this is approach:
If $domainsis empty, do only steps 2&3, otherwise:
foreach($domains as $d): get all the rows of Subdomains where Subdomain.domain_id = Domains.id as $subdomains
foreach($subdomains as $s) : go get all the rows of SubdomainsItems where SubdomainsItems.subdomain_id = Subdomains.id as $item_ids
foreach($items_ids as $i): get all the rows of Items where Items.id = SubdomainsItems.items_id
This works, but I think this is obviating the power of a relational database and I'd like to understand how this should be done (ie. according to either Cakephp convention or simply by whatever MySQL statement would achieve this).
Help would be hugely appreciated, I try to learn the more complex aspects of SQL but it just goes right over my head. :S
Understanding the necessary query
With the structure described in the question the kind of query necessary is of the form:
SELECT
*
FROM
items
LEFT JOIN
subdomains ON (
items.subdomain_id = subdomains.id
)
LEFT JOIN
domains ON (
subdomains.domain_id = domains.id
)
WHERE
domains.name = "foo"
AND
subdomains.name IN ('some', 'list', 'of', 'subdomains');
Compared to the logic in the question this joins all three tables together and permits finding all items by domain name, or subdomain name (or any other criteria involving any or all three tables); Generally speaking if you want to find data in a db and use more than one query to get it - there's a more efficient way to do it.
Implementing the find call
There are a number of ways of creating such a query with Cake. The simplest, probably, is to use the join key and just specify the joins explicitly:
function fetchWithin($domains = null, $subdomains = null) {
$params = array(
'joins' => array(
array('table' => 'subdomains',
'alias' => 'Subdomain',
'type' => 'LEFT',
'conditions' => array(
'Subdomain.id = Item.subdomain_id',
)
),
array('table' => 'domains',
'alias' => 'Domain',
'type' => 'LEFT',
'conditions' => array(
'Domain.id = Subdomain.domain_id',
)
)
)
);
if ($domains) { // single value or an array
$params['conditions']['Domain.name'] = $domains;
}
if ($subdomains) { // single value or an array
$params['conditions']['Subdomain.name'] = $subdomains;
}
return $this->find('all', $params);
}
I am not very good with DB queries. And with Yii it's more complicated, since I am not very used to it.
I need to optimize a simple query
$userCalendar = UserCalendar::model()->findByAttributes(array('user_id'=>$user->id));
$unplannedEvents = CalendarEvent::model()->findAllByAttributes(array('calendar_id'=> $userCalendar->calendar_id,'planned'=>0));
CalendarEvent table, i.e the second table from which I need records does not have an user_id but a calendar_id from which I could get user_id from UserCalendar, i.e. the first table hence I created a UserCalendar object which is not a very good way as far as I understand.
Q1. What could I do to make it into one.
Q2. Yii does this all internally but I want to know what query it built to try it seperately in MySQL(phpMyAdmin), is there a way to do that?
Thanks.
Q1: You need to have the relation between UserCalendar and CalendarEvent defined in both of your active record models (in the method "relations").
Based on your comments, it seems like you have the Calendar model that has CalendarEvent models and UserCalendar models.
Lets assume your relations in Calendar are:
relations() {
return array(
'userCalendar' => array(self::HAS_MANY, 'UserCalendar', 'calendar_id'),
'calendarEvent' => array(self::HAS_MANY, 'CalendarEvent', 'calendar_id'),
}
In CalendarEvent:
relations() {
return array( 'calendar' => array(self::BELONGS_TO, 'Calendar', 'calendar_id'), );
}
And in UserCalendar:
relations() {
return array( 'calendar' => array(self::BELONGS_TO, 'Calendar', 'calendar_id'), );
}
So to make the link between UserCalendar and CalendarEvent you'll need to use Calendar
$criteria = new CDbCriteria;
$criteria->with = array(
"calendarEvent"=>array('condition'=>'planned = 0'),
"userCalendar"=>array('condition'=> 'user_id =' . $user->id),
);
$calendar = Calendar::model()->find($criteria);
and $calendar->calendarEvent will return an array of calendarEvent belonging to the user
Q2: you can enable web logging so all the db request (and others stuffs) will appear at the end of your page:
Logging in Yii (see CWebLogging)
In your application configuration put
'components'=>array(
......
'log'=>array(
'class'=>'CLogRouter',
'routes'=>array(
array(
'class'=>'CWebLogRoute',
),
),
),
),
I just created a data table based on a query and displayed it successfully using theme_table().
Now, I'd like to add some filters to the table but have no idea how to proceed.
Is there a built-in feature that allow me to do this easily, or should I manually add a form and update the query/redisplay the results each time the user selects something?
Thanks for your help!
I think you want to use pager_query and tablesort_sql: it's especially made for creating tables of data with pagination and sorting capabilities (and themes usually theme such tables nicely out of the box).
Example:
<?php
// The regular query without sorting or pagination parameters
$sql = 'SELECT cid, first_name, last_name, company, city FROM {clients}';
// Number of rows per page
$limit = 20;
// List of table columns ("field" is the matching database column from the sql query)
$header = array(
array('data' => t('Name'), 'field' => 'last_name', 'sort' => 'asc'),
array('data' => t('Company'), 'field' => 'company'),
array('data' => t('City'), 'field' => 'city')
);
// Calculates how to modify the SQL query according to the current pagination and sorting settings
// Then performs the database query
$tablesort = tablesort_sql($header);
$result = pager_query($sql . $tablesort, $limit);
$rows = array();
while ($client = db_fetch_object($result)) {
$rows[] = array(l($client->last_name.', '.$client->first_name, 'client/'.$client->cid), $client->company, $client->city);
}
// A message in case no results were found
if (!$rows) {
$rows[] = array(array('data' => t('No client accounts created yet.'), 'colspan' => 3));
}
// Then you can pass the data to the theme functions
$output .= theme('table', $header, $rows);
$output .= theme('pager', NULL, $limit, 0);
// And return the HTML output
print $output;
?>
(I added comments, but the original version of the example comes from this page)
Alternatively, maybe you don't need to make a module at all if you're just trying to make a page that displays a list of data, you may prefer using the Views module.