i'm trying to do something perhaps a bit too crazy with Eloquent right now, i have a database where i have the following Tables
Crons - (Has Many) - Campaign - (Has Many) - Leads - (Has Many) - Conversions
I need to get all leads from a Cron, that have no entries in the Conversions table in the last X amount of days
I'm thinking of using a Scope on the Cron model but i'm completely stuck on how to proceed from here.
public function scopeWithValidLeads($query) {
return $query->with(['leads' => function($q) {
}]);
}
So i need to get LEADS where the following is true.
A - The leads belong to a campaign associated with the Cron via a Many-To-Many relationship.
B - They have no record in the conversions table Under this specific campaign or if they do, that the lead is older than X amount of days.
You can get your desired result using doesntHave() method like this:
$x = 10; // last 10 days
$crons = Cron::doesntHave('compaign.leads.conversions')
->whereBetween('created_at', [Carbon::now(), Carbon::now()->addDays($x)])
->get();
Querying Relationship Absence: When accessing the records for a model, you may wish to limit your results based on the absence of a relationship. For example, imagine you want to retrieve all blog posts that don't have any comments. To do so, you may pass the name of the relationship to the doesntHave method
UPDATE
As per the updated question conditions, according to my understanding the leads can be obtained by:
$leads = Lead::whereHas('compaign', function($q) use($compaign) {
$q->has('crons')
->where('id', $compaign->id);
// Use the above line if in case of a compaign is to be filtered out
})->doesntHave('conversions')
->whereBetween('created_at', [Carbon::now(), Carbon::now()->addDays($x)])
->get();
Hope this helps!
Related
Circumstances
I have three models/db-tables related with 1:n each: An order has multiple commissions and a commission has multiple commission_positions. Therefore the commission_position has an FK-field containing a commission id. The commission itself has an FK-field containing the id of an order.
Order > Commission > CommissionPositions
Problem
What I need to do now is select all the CommissionPositions having a certain value in the related Order-Model. Obvious solution is to use the Query-Object of CommissionPosition which I extended with a named scope. The named scope looks like this:
class CommissionPositionQuery extends ActiveQuery
{
/**
* Named scope to filter positions of a certain alpha order id
* #param integer $id the alpha order id
* #return \common\models\query\CommissionPositionQuery
*/
public function alphaOrderId($id)
{
//TODO: with not working
$this->with(['commission.order']);
$this->andWhere(['alpha_order_id'=>$id]);
return $this;
}
}
The relation commission is defined on the Commission-Model and working. The second relation order is defined on the commission-model and working as well. The filtered field alpha_order_id is in the Order-Table. Now I execute the query like this:
$filteredPositions = CommissionPosition::find()->alphaOrderId(17)->all();
The scope is called successfully and the where-part is used, but when I check the generated SQL I see no join-statements even though I use the with-method to tell yii to fetch the relation together. The response is 'unknown column alpha_order_id' which makes sense as there is no join to the related tables. This is the generated SQL:
SELECT * FROM `commission_position` WHERE (`alpha_order_id`=17)
What am I missing? Is this a bug of Yii2?
Found the soution myself. The naming changes between Yii1 and Yii2 lead to a little confusion. To prevent others from wasting time here the details:
Yii1
In yii 1 you would join in a relation (exemplary: commission) directly like this:
$query->with = 'commission'
$query->together = true;
Yii2 / difference
When calling the with-method like showed in the question the relation was successfully added to the with-array of the ActiveQuery. However, when executing the query, the join part was missing.
Solution
Seems like the with-method is NOT the way to go. Instead I used the method called joinWith with the following signature:
public function joinWith($with, $eagerLoading = true, $joinType = 'LEFT JOIN')
Now as described in the answer I defined the relation as the first argument ('commission.order') and left the rest as is, because the default values are perfectly fine. Pay attention to the default value of the second parameter. this makes sure the relations are joined in directly!
Voilà...the resulting sql contains the needed joins. One pitfall is to be considered though: Ambigious column namings is of course to be handled by ourselves! Link to the documentation of the method:
http://www.yiiframework.com/doc-2.0/yii-db-activequery.html#joinWith()-detail
If you want a JOIN use:
$this->joinWith(['commission.order']);
I have two models (Organizations and Interactions) and I'd like to query the Organization model for all of the Orgs that have no Interactions. Organizations have a one-to-many relationship with Interactions.
I tried looking into anti-joins in raw SQL, but got nowhere. I also wanted to totally avoid anything like getting all of the full Organizations, then iterating through them to check to see if they had any Interactions, because that's completely impractical given the amount of data I'm working with.
To clarify, I want to avoid this:
$organizations = Organization::all();
foreach ($organizations as $org)
if($org->interactions()->count() == 0){
//Add the org to an array for later use because it has no interactions
}
I'm using Laravel 3.x, and I can't upgrade because the project is really big and I don't have the month it would take to upgrade to 4.1 right now. If there's a significantly better way to do stuff like this 4, that would make selling the conversion process easier.
Here's some relevant code:
//From organization.php
public function interactions() {
return $this->has_many('Interaction');
}
//From interaction.php
public function organization() {
return $this->belongs_to('Organization');
}
// select all Organization IDs that have at least 1 interaction
$uniqueOrganizationIDs = DB::raw('SELECT organization_id FROM interactions GROUP BY(organization_id)');
// Select orgs that were not in the above list.
Organization::whereNotIn('id', $uniqueOrganizationIDs)->get();
This is the solution I came up with:
Query the Organization and Interaction models using list(). For Orgs, get their ID. For Interactions, get their organization_id. I figure these are two low-footprint, fast queries.
Do an array_diff() on them to get an array of Organizations that don't have Interactions.
Query Organization using where_in(), feeding it the diff'ed array.
It looks like this:
$organizationIDs = DB::table('organizations')->where('is_deleted', '=', 0)->lists('id');
$interactionIDs = DB::table('interactions')where('is_deleted', '=', 0)->lists('organization_id');
$uncontactedOrganizationIDs = array_diff($organizationIDs, $interactionIDs);
$uncontactedOrganizations = Organization::where_in('id', $uncontactedOrganizationIDs)->order_by('created_at', 'DESC')->get();
Is there a better way to do this? I feel like there has to be.
I'm writing a simple forum for a php site. I'm trying to calculate the post counts for each category. Now a category can belong to another category with root categories being defined as having a NULL parent_category_id. With this architecture a category can have an unlimited number of sub-categories and keeps the table structure fairly simple.
To keep things simple lets say the categories table has 3 fields: category_id, parent_category_id, post_count. I don't think the remaining database structure is relevant so I'll leave it out for now.
Another trigger is calling the categories table causing this trigger to run. What I want is it to update the post count and then recursively go through each parent category increasing that post count.
DELIMITER $$
CREATE TRIGGER trg_update_category_category_post_count BEFORE UPDATE ON categories FOR EACH ROW
BEGIN
IF OLD.post_count != NEW.post_count THEN
IF OLD.post_count < NEW.post_count THEN
UPDATE categories SET post_count = post_count + 1 WHERE categories.category_id = NEW.parent_category_id;
ELSEIF OLD.post_count > NEW.post_count THEN
UPDATE categories SET post_count = post_count - 1 WHERE categories.category_id = NEW.parent_category_id;
END IF;
END IF;
END $$
DELIMITER ;
The error I'm getting is:
#1442 - Can't update table 'categories' in stored function/trigger because it is already used by statement which invoked this stored function/trigger.
I figure you can do a count() on each page load to calculate the total posts but on large forums this will slow things down as discussed many times on here (e.g. Count posts with php or store in database). Therefore for future proofing i'm storing the post count in the table. To go one step further I thought i'd use triggers to update these counts rather than PHP.
I understand there are limitations in MySQL for running triggers on the same table that's being updated which is what is causing this error (i.e. to stop an infinite loop) but in this case surely the loop would stop once it reaches a category with a NULL parent_category_id? There must be some kind of solution whether it's adjusting this trigger or something different entirely. Thanks.
EDIT I appreciate this might not be the best way of doing things but it is the best thing I can think of. I suppose if you changed a parents category to another it would mess things up, but this could be fixed by another trigger which re-syncs everything. I'm open to other suggestions on how to solve this problem.
I usually recommend against using triggers unless you really, really need to; recursive triggers are a great way of introducing bugs that are really hard to reproduce, and require developers to understand the side effects of an apparently simple action - "all I did was insert a record into the categories table, and now the whole database has locked up". I've seen this happen several times - nobody did anything wrong or stupid, it's just a risk you run with side effects.
So, I would only resort to triggers once you can prove you need to; rather than relying on the opinion of strangers based on generalities, I'd rig up a test environment, drop in a few million test records, and try to optimize the "calculate posts on page load" solution so it works.
A database design that might help with that is Joe Celko's "nested set" schema - this takes a while to get your head round, but can be very fast for querying.
Only once you know you have a problem that you really can't solve other than by pre-computing the post count would I consider a trigger-based approach. I'd separate out the "post counts" into a separate table; that keeps your design a little cleaner, and should get round the recursive trigger issue.
The easiest solution is to fetch all the posts per category and afterwards link them together using a script/programming language:
for instance in php:
<?php
// category: id, parent, name
// posts: id, title, message
$sql = "select *, count(posts.id) From category left join posts ON posts.cat = category.id Group by category.id";
$query = mysql_query($sql);
$result = array();
while($row = mysql_fetch_assoc($query)){
$parent = $row['parent'] == null ? 0 : $row['parent'];
$result[$parent][] = $row;
}
recur_count(0);
var_dump($result);
function recur_count($depth){
global $result;
var_dump($result[$depth],$depth);
foreach($result[$depth] as $id => $o){
$count = $o['count'];
if(isset($result[$o['id']])){
$result[$depth][$id]['count'] += recur_count($o['id']);
}
}
return $count;
}
Ok so for anyone wondering how I solved this I used a mixture of both triggers and PHP.
Instead of getting each category to update it's parent, I've left it to the following structure: a post updates it's thread and then a thread updates it's category with the post count.
I've then used PHP to pull all categories from the database and loop through adding up each post count value using something like this:
function recursiveCategoryCount($categories)
{
$count = $categories['category']->post_count;
if(!is_null($categories['children']))
foreach($categories['children'] as $child)
$count += recursiveCategoryCount($child);
return $count;
}
At worst instead of PHP adding up every post on every page load, it only adds up the total category posts (depending at what node in the tree you are in). This should be very efficient as you're reducing the total calculations from 1000s to 10s or 100s depending on your number of categories. I would also recommend running a script every week to recalculate the post counts in case they become out of sync, much like phpBB. If I run into issues using triggers then I'll move that functionality into the code. Thanks for everyones suggestions.
This is an efficiency/best practice question. Hoping to receive some feed back on performance. Any advice is greatly appreciated.
So here is a little background in what i have setup. I'm using codeigniter, the basic setup is pretty similar to any other product relationships. Basic tables are: Brands, products, categories. On top of these tables there is a need for install sheets, marketing materials, and colors.
I created some relationship tables:
Brands_Products
Products_Colors
Products_Images
Products_Sheets
I also have a Categories_Relationships table that holds all of the relationships to categories. Install sheets etc can have their own categories but i didn't want to define a different category relationship table for each type because i didn't think that would be very expandable.
On the front end I am sorting by brands, and categories.
I think that covers the background now to the efficiency part. I guess my question pertains mostly to weather it would be better to use joins or to make separate calls to return individual parts of each item (colors, images, etc)
What I currently have coded is working, and sorting fine but I think i can improve the performance, as it take some time to return the query. Right now its returning about 45 items. Here is my first function it grabs all the products and its info.
It works by first selecting all the products and joining it's brand information. then looping through the result i set up the basic information, but for the categories images and installs i am using functions that returns each of respected items.
public function all()
{
$q = $this->db
->select('*')
->from('Products')
->join('Brands_Products', 'Brands_Products.product_id = Products.id')
->join('Brands', 'Brands.id = Brands_Products.brand_id')
->get();
foreach($q->result() as $row)
{
// Set Regular Data
$data['Id'] = $row->product_id;
$data['Name'] = $row->product_name;
$data['Description'] = $row->description;
$data['Brand'] = $row->brand_name;
$data['Category'] = $this->categories($row->product_id);
$data['Product_Images'] = $this->product_images($row->product_id);
$data['Product_Installs'] = $this->product_installs($row->product_id);
$data['Slug'] = $row->slug;
// Set new item in return object with created data
$r[] = (object)$data;
}
return $r;
}
Here is an example of one of the functions used to get the individual parts.
private function product_installs($id)
{
// Select Install Images
$install_images = $this->db
->select('*')
->where('product_id', $id)
->from('Products_Installs')
->join('Files', 'Files.id = Products_Installs.file_id')
->get();
// Add categories to category object
foreach($install_images->result() as $pImage)
{
$data[] = array(
'id' => $pImage->file_id,
'src' => $pImage->src,
'title' => $pImage->title,
'alt' => $pImage->alt
);
}
// Make sure data exists
if(!isset($data))
{
$data = array();
}
return $data;
}
So again really just looking on advice on what is the most efficient, best practice way of doing this. I really appreciate any advice, or information.
I think your approach is correct. There are only a couple of options: 1) load your product list first, then loop, and load required data for each product row. 2) create a big join on all tables first, then loop through (possibly massive) cartesian product. The second might get rather ugly to parse. For example, if you got Product A and Product B, and Product A has Install 1, Install 2, Install 3, and product B has Install 1, and Install 2,t hen your result is
Product A Install 1
Product A Install 2
Product A Install 3
Product B Install 1
Product B Install 2
Now, add your images and categories to the join and it might become huge.
I am not sure what the sizes of your tables are but returning 45 rows shouldn't take long. The obvious thing to ensure (and you probably did that already) is that product_id is indexed in all tables as well as your brands_products tables and others. Otherwise, you'll do a table scan.
The next question is how you're displaying your data on the screen. So you're getting all products. Do you need to load categories, images, installs when you're getting a list of products? If you're simply listing products on the screen, you might want to wait to load that data until user picks a products they are viewing.
On a side note, any reason you're converting your array to object
$r[] = (object)$data;
Also, in the second function, you can simply add
$data = array();
before the foreach, instead of
// Make sure data exists
if(!isset($data))
{
$data = array();
}
You can try this:
Query all of the products
Get all of the product IDs from step 1
Query all of the install images that has a product ID from step 2, sorted by product ID
Iterate through the products from step 1, and add the results from step 3
That takes you from 46 queries (for 45 products) to 2 queries, without any additional joins.
You can also use CodeIgniter's Query Caching to increase performance even further, if it's worth the time to write the code to reset the cache when data is updated.
Doing more work in PHP is generally better than doing the work in MySQL in terms of scalability. You can scale PHP easily with load balancers and web servers. Scaling MySQL isn't as easy due to concurrency issues.
I have a datamodel, let's say: invoices (m:n) invoice_items
and currently I store the invoice total, calculated in PHP by totalling invoice_items, in a column in invoices. I don't like storing derived data as it paves the way for errors later.
How can I create a logical column in the invoices table in MySql? Is this something I would be better handling in the PHP (in this case CakePHP)?
There's something called Virtual Fields in CakePHP which allows you to achieve the same result from within your Model instead of relying on support from MySQL. Virtual Fields allow you to "mashup" various data within your model and provide that as an additional column in your record. It's cleaner than the other approaches here...(no afterFind() hacking).
Read more here: http://book.cakephp.org/view/1608/Virtual-fields
Leo,
One thing you could do is to modify the afterFind() method in your model. This would recalculate the total any time you retrieve an invoice (costing runtime processing), but would mean you're not storing it in the invoices table, which is apparently what you want to avoid (correct if I'm wrong).
Try this:
class Invoice extends AppModel {
// .. other stuff
function afterFind() {
parent::afterFind();
$total = 0;
foreach( $this->data['Invoice']['InvoiceItems'] as $item )
$total += ($item['cost'] * $item['quantity']);
$this->data['Invoice']['total'] = $total;
}
}
I may have messed up the arrays on the hasMany relationship (the foreach line), but I hope you get the jist of it. HTH,
Travis
Either you can return the derived one when you want it via
SELECT COUNT(1) as total FROM invoice_items
Or if invoices can be multiple,
//assuming that invoice_items.num is how many there are per row
SELECT SUM(num) as total FROM invoice_items
Or you can use a VIEW, if you have a certain way you want it represented all the time.
http://forge.mysql.com/wiki/MySQL_virtual_columns_preview
It's not implemented yet, but it should be implemented in mysql 6.0
Currently you could create a view.