I have 150000 thousand records on my database. I cannot use the paginate of the laravel because I am using vue cli so this is my approach on my api
$jobsTotal = $jobs->select('jobs.id as job_id')
->join('customer','customer.id','jobs.customer_id')
->join('address','address.v2_id','jobs.service_location_id')
->where('jobs.location_id', $data->location_id)->get()->toArray();
$jobs = UserHelper::paginate($jobsTotal , $data->pageSize, $data->page)
And this is my helper function
public function paginate($array,$per_page, $page)
{
$total = count($array);
$current_page = $page ?? 1;
$starting_point = ($current_page * $per_page) - $per_page;
$array = array_slice($array, $starting_point, $per_page);
$paginate_result = new Paginator($array, $total, $per_page, $current_page);
return $paginate_result;
}
what I want is count the total jobs that is 150000 and wont take much time to load
Related
We have a large MySQL Table (say) 50,000 records or rows. The loading time of all the records is close to 20 mins. By loading I mean the data getting displayed in the browser or in a Javascript Grid - time taken to load this data is around 20 mins. This is creating a big problem for us. We have even tried to return all data in JSON following this approach How to convert result table to JSON array in MySQL still we are facing a prolonged delay in loading.
The query is very much straight forward as shown below. Can anyone help us with any solution to overcome this problem. Thanks in advance!
<?php
include 'includes/xxx.php';
$returnArray = array();
$totalSymptoms = 0;
$matchedSymptomIds = array();
$cutOff = 10;
$runningGlobalSymptomId = "";
$symptomResult = mysqli_query($db,"SELECT * FROM comparison_small WHERE (matched_percentage IS NULL OR matched_percentage >= ".$cutOff.")");
if(mysqli_num_rows($symptomResult) > 0){
while($symRow = mysqli_fetch_array($symptomResult)){
$dataArray = array();
$totalSymptoms++;
if($symRow['current_symptom'] == '1'){
// Global symptom
$runningGlobalSymptomId = $symRow['symptom_id'];
$dataArray['symptom_id'] = $symRow['symptom_id'];
$dataArray['row_id'] = "row".$symRow['symptom_id'];
$dataArray['symptom'] = $symRow['symptom'];
$dataArray['match'] = "";
}else{
// Local symptom
array_push($matchedSymptomIds, $symRow['symptom_id']);
$dataArray['symptom_id'] = $symRow['symptom_id'];
$dataArray['row_id'] = "row".$runningGlobalSymptomId."_".$symRow['symptom_id'];
$dataArray['symptom'] = $symRow['symptom'];
$dataArray['match'] = $symRow['matched_percentage'];
}
$returnArray[] = $dataArray;
}
}
I am trying to get the details from database and written below Laravel code to fetch the details from two table using join query. But it is taking too much time while fetching the data.
Can you guys help me? So it will take minimum time to fetch the API.
public function getWebsiteVisitorsDetails($businessId, $startTime, $endTime, $filterType){
$chatBotClicked = DB::table('chatbot_website_visitor_detail')
->join('website_visitors_task', 'website_visitors_task.website_session_id', '=', 'chatbot_website_visitor_detail.website_session_id')
->where('website_visitors_task.session_task_perform' , 3)
->where('chatbot_website_visitor_detail.business_id', $businessId);
if($filterType != ApiConstant::ALL_TIME){
$chatBotClicked = $chatBotClicked->whereBetween('website_visitors_task.created_at', [$startTime, $endTime]);
}
$feedbackClicked = DB::table('chatbot_website_visitor_detail')
->join('website_visitors_task', 'website_visitors_task.website_session_id', '=', 'chatbot_website_visitor_detail.website_session_id')
->where('website_visitors_task.session_task_perform' , 2)
->where('chatbot_website_visitor_detail.business_id', $businessId);
if($filterType != ApiConstant::ALL_TIME){
$feedbackClicked = $feedbackClicked->whereBetween('website_visitors_task.created_at', [$startTime, $endTime]);
}
$firstTimeVisitor = DB::table('chatbot_website_visitor_detail')
->select('chatbot_website_visitor_detail.id')
->where('chatbot_website_visitor_detail.business_id', $businessId)
->where('chatbot_website_visitor_detail.from_popup_widget', 1);
if($filterType != ApiConstant::ALL_TIME){
$firstTimeVisitor = $firstTimeVisitor->whereBetween('chatbot_website_visitor_detail.created_at', [$startTime, $endTime]);
}
$returnData['chatbot_clicked'] = $chatBotClicked->count();
$returnData['feedback_clicked'] = $feedbackClicked->count();
$returnData['first_time_user'] = $firstTimeVisitor->count();
}
I need to save data from an order table from one database to another through Laravel. I created a function in my controller like this:
public function getMarketplace()
{
$orderoc = OrderOC::orderBy('oc_order.date_added', 'desc')
->join('oc_order_product', 'oc_order_product.order_id', 'oc_order.order_id')
->join('oc_order_history', 'oc_order_history.order_id', 'oc_order.order_id')
->where('oc_order_history.order_status_id', '=', '17')
->get();
foreach($orderoc as $oc){
$ordererp = new Order;
$ordererp->erp_createdid = $oc->created_id;
$ordererp->erp_marketplaceid = 1;
$ordererp->erp_site = rand(1,100000000);
$ordererp->erp_payment_method = $oc->payment_method;
$ordererp->erp_orderdate = $oc->date_added;
$ordererp->erp_orderaprove = $oc->date_added;
$ordererp->erp_billingid = 1;
$ordererp->erp_shippingid = 1;
$ordererp->erp_marketplace = 'Comércio Urbano';
$ordererp->erp_orderquantity = $oc->quantity;
$ordererp->erp_erro = '';
$ordererp->erp_product_ok = 1;
$ordererp->erp_compraId = null;
$ordererp->save();
if(strlen($oc->created_id) == 0){
$oc->created_id = rand(1,10000000);
$oc->save();
}
$orderprod = new OrderProduct;
$orderprod->erp_productid = $oc->product_id;
$orderprod->erp_createdid = $oc->created_id;
$orderprod->erp_model = $oc->model;
$orderprod->erp_quantity = $oc->quantity;
}
}
One table is from my ERP and the other is responsible for receiving OpenCart purchases, but every time I run, the same product appears more than once in my order table.
(It is possible to see through the purchase date, since created_id is created in the controller function)
Does anyone know how to tell me why data is duplicated when inserted inside a foreach? This is not the first time, if you tell me a more robust way of doing the job, I'm grateful. Any suggestion? Thank you in advance!
One possiblity is you put unique validation on a table that receive the data
This is my database in which I want to calculate total
$this->db->select('*');
$this->db->select_sum('total_sale');
$query = $this->db->get('one_month_report');
if ($query->num_rows() > 0) {
return $query->result();
} else {
return FALSE;
}
}
if you want ot select sum. make like documentation
https://www.codeigniter.com/userguide3/database/query_builder.html
$this->db->select('(SELECT SUM(payments.amount) FROM payments WHERE payments.invoice_id=4') AS amount_paid', FALSE);
$query = $this->db->get('mytable');
Use the NumberFormatter class to parse a locale-specific number.
but for using it you need PHP version >= 5.3 And intl extension version >= 1.0
if you have it, uncomment this line extension=php_intl.dll from php.ini if it is commented.
and use it as below example:
<?php
$num1 = '12,478.76';
$nf = new NumberFormatter("en_EN", NumberFormatter::DECIMAL);
var_dump($nf->parse($num1));
// output: float(12478.76)
// it will automatically parse int, double,float etc.
?>
getChildIds can be very slow if running it against a large set of resources, so I am trying to write a query and some code to get all the child ids faster.
However I am getting different results from getChildIds & my script.
Can anyone see why these would be yielding different results?
Method using getChildIDs:
$parentDepth = isset($scriptProperties['parentDepth']) ? $scriptProperties['parentDepth'] : 5;
$parents = explode(',', $parents);
$children = array();
foreach ($parents as $parent){
$ids = $modx->getChildIds($parent,10,array('context' => 'web'));
foreach($ids as $id){
$children[] = $id;
}
}
echo ' number of children = ' . count($children);
method using queriees & a loop:
$comma_separated = implode(",", $parents);
$sql = "SELECT `id` from modx_site_content where `parent` IN (".$comma_separated.") and published = 1 and isfolder = 0 and deleted = 0 and hidemenu = 0;";
$results = $modx->query($sql);
$mychildren = array();
while ($row = $results->fetch(PDO::FETCH_ASSOC)) {
$mychildren[] = $row['id'];
}
for($i=0; $i <= 10; $i++){
$comma_separated = implode(",", $mychildren);
$sql = "SELECT `id` from modx_site_content where `parent` IN (".$comma_separated.") and published = 1 and isfolder = 0 and deleted = 0 and hidemenu = 0;";
$results = $modx->query($sql);
while ($row = $results->fetch(PDO::FETCH_ASSOC)) {
$mychildren[] = $row['id'];
}
}
echo ' number of children = ' . count($mychildren);
The getChildIDs method takes nearly 1.5 seconds to run and gives about 1100 results
The SQL/script method runs un under 0.1 second and gives 1700 results.
Either I'm not appending the child ids to the array properly ~or~ maybe getChildIDs is filtering out some other results?
does anyone have any clues as to what could be happening here?
You can try to use built-in method of pdoFetch.
$pdo = $modx->getService('pdoFetch');
$ids = $pdo->getChildIds('modResource', 0);
print_r($ids);
It also recursive, but can be better in some situations.
Of course, you need to install pdoTools from the repository first.
Looking at the code again, the discrepancy in results is pretty obvious now. I'm appending results to the child id array, it's inserting duplicates since one parent can have many children.
the solution to avoid duplicates:
$mychildren[$row['id']] = $row['id'];
getChildIds - is recursive, so it slower by default.