SQL SELECT query excluding an array of id's for infinite scroll [duplicate] - mysql

I have followed help located in this topic: Using infinite scroll w/ a MySQL Database
And have gotten close to getting this working properly. I have a page that is displayed in blocks using jquery masonry, in which the blocks are populated by data from a mysql database. When I scroll to the end of the page I successfully get the loading.gif image but immediately after the image it says "No more posts to show." which is what it should say if that were true. I am only calling in 5 posts initially out of about 10-15, so the rest of the posts should load when I reach the bottom of the page but I get the message that is supposed to come up when there really aren't any more posts.
Here is my javascript:
var loading = false;
$(window).scroll(function(){
if($(window).scrollTop() == $(document).height() - $(window).height()) {
var h = $('.blockContainer').height();
var st = $(window).scrollTop();
var trigger = h - 250;
if((st >= 0.2*h) && (!loading) && (h > 500)){
loading = true;
$('div#ajaxLoader').html('<img src="images/loading.gif" name="HireStarts Loading" title="HireStarts Loading" />');
$('div#ajaxLoader').show();
$.ajax({
url: "blocks.php?lastid=" + $(".masonryBlock:last").attr("id"),
success: function(html){
if(html){
$(".blockContainer").append(html);
$('div#ajaxLoader').hide();
}else{
$('div#ajaxLoader').html('<center><b>No more posts to show.</b></center>');
}
}
});
}
}
});
Here is the php on the page the blocks are actually on. This page initially posts 5 items from the database. The javascript grabs the last posted id and sends that via ajax to the blocks.php script, which then uses the last posted id to grab the rest of the items from the database.
$allPosts = $link->query("/*qc=on*/SELECT * FROM all_posts ORDER BY post_id DESC LIMIT 5");
while($allRows = mysqli_fetch_assoc($allPosts)) {
$postID = $link->real_escape_string(intval($allRows['post_id']));
$isBlog = $link->real_escape_string(intval($allRows['blog']));
$isJob = $link->real_escape_string(intval($allRows['job']));
$isVid = $link->real_escape_string(intval($allRows['video']));
$itemID = $link->real_escape_string(intval($allRows['item_id']));
if($isBlog === '1') {
$query = "SELECT * FROM blogs WHERE blog_id = '".$itemID."' ORDER BY blog_id DESC";
$result = $link->query($query);
while($blogRow = mysqli_fetch_assoc($result)) {
$blogID = $link->real_escape_string($blogRow['blog_id']);
$blogTitle = $link->real_escape_string(html_entity_decode($blogRow['blog_title']));
$blogDate = $blogRow['pub_date'];
$blogPhoto = $link->real_escape_string($blogRow['image']);
$blogAuthor = $link->real_escape_string($blowRow['author']);
$blogContent = $link->real_escape_string($blogRow['content']);
//clean up the text
$blogTitle = stripslashes($blogTitle);
$blogContent = html_entity_decode(stripslashes(truncate($blogContent, 150)));
echo "<div class='masonryBlock' id='".$postID."'>";
echo "<a href='post.php?id=".$blogID."'>";
echo "<div class='imgholder'><img src='uploads/blogs/photos/".$blogPhoto."'></div>";
echo "<strong>".$blogTitle."</strong>";
echo "<p>".$blogContent."</p>";
echo "</a>";
echo "</div>";
}
}
Here is the php from the blocks.php script that the AJAX calls:
//if there is a query in the URL
if(isset($_GET['lastid'])) {
//get the starting ID from the URL
$startID = $link->real_escape_string(intval($_GET['lastid']));
//make the query, querying 25 fields per run
$result = $link->query("SELECT * FROM all_posts ORDER BY post_id DESC LIMIT '".$startID."', 25");
$html = '';
//put the table rows into variables
while($allRows = mysqli_fetch_assoc($result)) {
$postID = $link->real_escape_string(intval($allRows['post_id']));
$isBlog = $link->real_escape_string(intval($allRows['blog']));
$isJob = $link->real_escape_string(intval($allRows['job']));
$isVid = $link->real_escape_string(intval($allRows['video']));
$itemID = $link->real_escape_string(intval($allRows['item_id']));
//if the entry is a blog
if($isBlog === '1') {
$query = "SELECT * FROM blogs WHERE blog_id = '".$itemID."' ORDER BY blog_id DESC";
$result = $link->query($query);
while($blogRow = mysqli_fetch_assoc($result)) {
$blogID = $link->real_escape_string($blogRow['blog_id']);
$blogTitle = $link->real_escape_string(html_entity_decode($blogRow['blog_title']));
$blogDate = $blogRow['pub_date'];
$blogPhoto = $link->real_escape_string($blogRow['image']);
$blogAuthor = $link->real_escape_string($blowRow['author']);
$blogContent = $link->real_escape_string($blogRow['content']);
$blogTitle = stripslashes($blogTitle);
$blogContent = html_entity_decode(stripslashes(truncate($blogContent, 150)));
$html .="<div class='masonryBlock' id='".$postID."'>
<a href='post.php?id=".$blogID."'>
<div class='imgholder'><img src='uploads/blogs/photos/".$blogPhoto."'></div>
<strong>".$blogTitle."</strong>
<p>".$blogContent."</p>
</a></div>";
}
}
echo $html;
}
I have tried using the jquery infinite-scroll plugin, but it seemed much more difficult to do it that way. I don't know what the issue is here. I have added alerts and did testing and the javascript script is fully processing, so it must be with blocks.php right?
EDIT: I have made a temporary fix to this issue by changing the sql query to SELECT * FROM all_posts WHERE post_id < '".$startID."' ORDER BY post_id DESC LIMIT 15
The blocks are now loading via ajax, however they are only loading one block at a time. The ajax is sending a request for every single block and they are fading in one after another, is it possible to make them all fade in at once with jquery masonry?

I seen your code in another answer, and I would recommend using the LIMIT functionality in MySql instead of offsetting the values. Example:
SELECT * FROM all_posts ORDER BY post_id DESC LIMIT '".(((int)$page)*5)."',5
This will just take a page number in the AJAX request and get the offset automatically. It's one consistent query, and works independent of the last results on the page. Send something like page=1 or page=2 in your jQuery code. This can be done a couple different ways.
First, count the number of elements constructed on the page and divide by the number on the page. This will yield a page number.
Second, you can use jQuery and bind the current page number to the body:
$(body).data('page', 1)
Increment it by one each page load.
Doing this is really the better way to go, because it uses one query for all of the operations, and doesn't require a whole lot of information about the data already on the page.
Only thing to note is that this logic requires the first page request to be 0, not 1. This is because 1*5 will evaluate to 5, skipping the first 5 rows. If its 0, it will evaluate to 0*5 and skip the first 0 rows (since 0*5 is 0).
Let me know any questions you have!

Have you tried doing any debugging?
If you are not already using, I would recommend getting the firebug plugin.
Does the ajax call return empty? If it does, try echoing the sql and verify that is the correct statement and that all the variables contain the expected information. A lot of things could fail considering there's a lot of communication happening between client, server and db.
In response to your comment, you are adding the html in this piece of code:
if(html){
$(".blockContainer").append(html);
$('div#ajaxLoader').hide();
}
I would do a console.log(html) and console.log($(".blockContainer").length) before the if statement.

Related

Can't wrap my head around MySQL statement

I have two tables:
cache and main
In cache there are a lot of fields; in main a little less. A UNION is not going to work because of the unequal number of columns.
cache
client - file - target - many other columns
main
client - file - target - few other columns
From cache I would like all columns for which main.target LIKE '%string%', cache.client = main.client, cache.file = main.file
For these particular records, target, client and file are always the same in main and cache.
I just can't get my head around this, but then again MySQL never was my strongest point.
Thank you very much in advance!
In the end combining the two SELECT statements with a UNION made things very complicated, for the simple reason there were countless other queries, some without UNION, that in the end all had to be processed by the same end routine presenting the results. As this was only a one-time query and time wasn't really an issue, in the end I just used SELECT on the two different tables and then combined the results by checking if a certain field was present. If not, the remaining results had to be fetched from the cache table; if so, the remaining results had to be fetched from the main table.
I actually wonder whether this solution is faster, slower or just as fast.
if (!isset($row['current']))
{
$field = $row['field'];
$sqlcache = "SELECT * FROM " . $dbtable . " WHERE (client = '$sqlclient' AND file = '$sqlfile' AND field = '$field')";
$resultcache = $conn->query($sqlcache);
if (!$resultcache)
{
die($conn->error);
}
$rowcache = $resultcache->fetch_assoc();
$currenttarget = $rowcache['current'];
$context = $rowcache['context'];
$dirtysource = $rowcache['dirtysource'];
$stringid = $rowcache['stringid'];
$limit = $rowcache['maxlength'];
$locked = $rowcache['locked'];
$filei = $rowcache['filei'];
}
else
{
$currenttarget = $row['current'];
$context = $row['context'];
$dirtysource = $row['dirtysource'];
$stringid = $row['stringid'];
$limit = $row['maxlength'];
$locked = $row['locked'];
$filei = $row['filei'];
}

Problem with loading large MySQL Table into JSON

We have a large MySQL Table (say) 50,000 records or rows. The loading time of all the records is close to 20 mins. By loading I mean the data getting displayed in the browser or in a Javascript Grid - time taken to load this data is around 20 mins. This is creating a big problem for us. We have even tried to return all data in JSON following this approach How to convert result table to JSON array in MySQL still we are facing a prolonged delay in loading.
The query is very much straight forward as shown below. Can anyone help us with any solution to overcome this problem. Thanks in advance!
<?php
include 'includes/xxx.php';
$returnArray = array();
$totalSymptoms = 0;
$matchedSymptomIds = array();
$cutOff = 10;
$runningGlobalSymptomId = "";
$symptomResult = mysqli_query($db,"SELECT * FROM comparison_small WHERE (matched_percentage IS NULL OR matched_percentage >= ".$cutOff.")");
if(mysqli_num_rows($symptomResult) > 0){
while($symRow = mysqli_fetch_array($symptomResult)){
$dataArray = array();
$totalSymptoms++;
if($symRow['current_symptom'] == '1'){
// Global symptom
$runningGlobalSymptomId = $symRow['symptom_id'];
$dataArray['symptom_id'] = $symRow['symptom_id'];
$dataArray['row_id'] = "row".$symRow['symptom_id'];
$dataArray['symptom'] = $symRow['symptom'];
$dataArray['match'] = "";
}else{
// Local symptom
array_push($matchedSymptomIds, $symRow['symptom_id']);
$dataArray['symptom_id'] = $symRow['symptom_id'];
$dataArray['row_id'] = "row".$runningGlobalSymptomId."_".$symRow['symptom_id'];
$dataArray['symptom'] = $symRow['symptom'];
$dataArray['match'] = $symRow['matched_percentage'];
}
$returnArray[] = $dataArray;
}
}

How to get last inserted id with insert method in laravel

In my laravel project I am inserting multiple records at time with modelname::insert method. Now I want to get last inserted id of it.I read somewhere when you insert multiple records with single insert method and try to get the last_record_id it will gives you the first id of the last inserted query bunch. But my first question is how to get last record id with following code .If I am able to get first id of the bunch .I ll make other ids for other record by my own using incremental variable.
Code to insert multiple record
if(!empty($req->contract_name) && count($req->contract_name)>0)
{
for($i=0; $i<count($req->contract_name); $i++)
{
$contract_arr[$i]['client_id'] = $this->id;
$contract_arr[$i]['contract_name'] = $req->contract_name[$i];
$contract_arr[$i]['contract_code'] = $req->contract_code[$i];
$contract_arr[$i]['contract_type'] = $req->contract_type[$i];
$contract_arr[$i]['contract_ext_period'] = $req->contract_ext_period[$i];
$contract_arr[$i]['contract_email'] = $req->contract_email[$i];
$contract_arr[$i]['created_at'] = \Carbon\Carbon::now();
$contract_arr[$i]['updated_at'] = \Carbon\Carbon::now();
$contract_arr[$i]['created_by'] = Auth::user()->id;
$contract_arr[$i]['updated_by'] = Auth::user()->id;
if($req->startdate[$i] != ''){
$contract_arr[$i]['startdate'] = date('Y-m-d',strtotime($req->startdate[$i]));
}
if($req->enddate[$i] != ''){
$contract_arr[$i]['enddate'] = date('Y-m-d',strtotime($req->enddate[$i]));
}
}
if(!empty($contract_arr)){
Contract::insert($contract_arr);
}
}
You should be able to call it like this
$lastId = Contract::insert($contract_arr)->lastInsertId();
If i see right, you're using a Model. Direct inserting only shows an success boolean. Try this instead:
Contract::create($contract_arr)->getKey()

Display selected amount of records from database

I need a page that displays records from a database, sorted by their jobs. So the database holds different kind of persons with different jobs. For example, on the page "teacher" I just want to display all the teachers, not the other persons. I want to have 3 persons on a page and a button "previous" and "next" beneath it. If an user clicks the next-button I want the next 3 records to be shown. When the user reaches the last records, I need the next-button to disappear. Same goes for "previous".
What I have so far:
This piece of code creates a value named startrow.
if (!isset($_GET['startrow']) or !is_numeric($_GET['startrow'])) {
//give the value of the starting row 0 because nothing was found in URL
$startrow = 0;
//otherwise take the value from the URL
} else {
$startrow = (int)$_GET['startrow'];
}
The query I have:
$sql = mysql_query("SELECT * FROM $tbl_name WHERE jobs='teacher' LIMIT $startrow, 3")or
die(mysql_error());
$sql2 = "SELECT COUNT(*) AS TotalJobs FROM $tbl_name WHERE jobs='teacher'";
$result_count = mysql_query($sql2);
$count = mysql_fetch_array($result_count);
I created the $count for the if / else function in the next part (the part that doesn't seem to work the way I want it to):
if ($startrow < $count )
echo 'Next';
$prev = $startrow - 3;
//only print a "Previous" link if a "Next" was clicked
if ($prev >= 0)
echo 'Previous';
So the previous-button seems to work the way it should. The next doesn't. I've created the
if ($startrow < $count)
so basicly, if the value of startrow is smaller than the total number of records, it puts a next-button. But if I test this, it displays the next-button anyhow, no matter the value of startrow.
What am I missing here?
mysql_fetch_array returns an array containing the count, not the value itself. Use $count['TotalJobs'] instead.
Also, you shouldn't use the mysql extension anymore, instead use PDO or MySQLi.
Edit
You can change:
$count = mysql_fetch_array($result_count);
to
$countArr = mysql_fetch_array($result_count);
$count = $countArr['TotalJobs'];

MySQL + PHP pagination with no offset

Ok, here is the code i have for my pagination:
$SQL = "SELECT
cpc.product_id,
cp.product_internal_ref,
cp.product_name,
cpa.product_sale_price,
cpa.is_product_service,
cpa.product_service_price,
cpi.image_name,
cpi.image_ext
FROM catalog_products_categories cpc
JOIN catalog_products cp ON cp.product_id = cpc.product_id
JOIN catalog_products_attributes cpa ON cpa.product_id = cpc.product_id
LEFT JOIN catalog_products_images cpi ON cpi.product_id = cpc.product_id
WHERE cpc.category_id = ".$catID;
// PAGINATOR SECTION
if($paginatorVARS['paginatorACTION'] == "next") {
$SQL .= " AND cpc.product_id > ".$paginatorVARS['paginatorGOID']." ";
$SQL .= "GROUP BY cpc.product_id ORDER BY cpc.product_id ASC LIMIT ".$paginatorVARS['catalogPaginatorPAGEROWS'];
}
elseif($paginatorVARS['paginatorACTION'] == "prev") {
$SQL .= " AND cpc.product_id < ".$paginatorVARS['paginatorGOID']." ";
$SQL .= "GROUP BY cpc.product_id ORDER BY cpc.product_id DESC LIMIT ".$paginatorVARS['catalogPaginatorPAGEROWS'];
}
// END PAGINATOR SECTION
I used the method described here: http://www.slideshare.net/Eweaver/efficient-pagination-using-mysql but i can't seam to find a way to also sort by other columns like for example by cpa.product_sale_price. If i do : ORDER BY cpc.product_id ASC, cpa.product_sale_price DESC/ASC it will break the paginator next results i dont know what happens... Please help!!
not saying their stuff is perfect, but it is often a place to start, to consider if you really want to reinvent the wheel, or take their concepts and incorporate it into your new rounder wheel. take a look at
http://dev.sencha.com/deploy/ext-4.0.0/examples/grid/paging.html
and use fiddler to examine the http gets that are occuring during pagination, as well as changes in the sort context. when it starts out it brings back the http response header, json data with jsonp callback, and top node value at the end of the json data specifying record count=6679 / 50 per page meaning 134 pages total as calculated by the front-end. the record count can vary from call to call (from page to page) as rows are inserted and deleted.
the sort column gets passed in the query string just like the pages impact the limit start,total that goes into sql. this can be seen in fiddler output showing changes in pages or sort column requests (column name, and asc or desc):
GET /forum/topics-browse-remote.php?_dc=1369401925806&page=1&start=0&limit=50&sort=lastpost&dir=DESC&callback=Ext.data.JsonP.callback1 HTTP/1.1
GET /forum/topics-browse-remote.php?_dc=1369401977137&page=1&start=0&limit=50&sort=replycount&dir=ASC&callback=Ext.data.JsonP.callback2 HTTP/1.1
GET /forum/topics-browse-remote.php?_dc=1369401978355&page=1&start=0&limit=50&sort=replycount&dir=DESC&callback=Ext.data.JsonP.callback3 HTTP/1.1