I have a large DB, with 50000+ rows, i'm trying to get all rows but with skipping every 50 rows for example.
I tried this from Laravel documentation: Offset & Limit
$users = DB::table('users')->skip(10)->take(5)->get();
But this will only skip the first 10 rows and get the next 5 rows. I can't find Eloquent solution to this problem.
Has anybode solve this before?
Solved using raw query:
return DB::select(DB::raw('
SELECT dateTime, row1, row2
FROM (
SELECT #row := #row +1 AS rownum, dateTime, row1, row2
FROM (
SELECT #row :=0
) r, users
) ranked
WHERE rownum % 50 = 0'));
It's much faster solution then #disf.asia suggestion.
You have to use a loop.
$start = 0;
$skip = 50;
$take = 1;
$all = array();
do {
$partial = User::skip($start++ * $skip)->take($take)->get();
$all = array_merge($all, (array) $partial);
} while (count($partial) > 0);
this will take 1 row every 50skipped rows, so 1st, 51th, 100th ... till the end of the table
Related
I am dealing with a query that is causing some deadlocks, I believe because it's trying to SELECT from the same table it's updating. Regardless, this is a terribly clumsy way of doing this and I'd like to find a better way.
One obvious solution here is to pull the SELECT out into a separate query, but I'm hoping one of you SQL ninjas out there has a suggestion for a more elegant solution that could get this done in a single query with much less overhead.
SET #update_id := 0;
UPDATE msgstream
SET retryCount = retryCount + 1,
retryTime = TIMESTAMPADD(SECOND,?,NOW(3)),
messageid = (SELECT #update_id := messageid)
WHERE receiver = ?
AND isDelivered = 0
AND retryCount < 10
AND retryTime < NOW(3)
AND (
SELECT m.counter
FROM (
SELECT COUNT(messageid) AS counter
FROM msgstream
WHERE receiver = ?
AND isDelivered = 0
AND retryCount < 10
AND retryTime > NOW(3)
) AS m
) = 0
ORDER BY messageid LIMIT 1;
I need help optimizing the below querys for a recurrent calendar i've built.
if user fail to accomplish all task where date
This is the query i use inside a forech which fetched all dates that the current activity is active.
This is my current setup, which works, but is very slow.
Other string explained:
$today=date("Y-m-d");
$parts = explode($sepparator, $datespan);
$dayForDate2 = date("l", mktime(0, 0, 0, $parts[1], $parts[2], $parts[0]));
$week2 = strtotime($datespan);
$week2 = date("W", $week2);
if($week2&1) { $weektype2 = "3"; } # Odd week 1, 3, 5 ...
else { $weektype2 = "2"; } # Even week 2, 4, 6 ...
Query1:
$query1 = "SELECT date_from, date_to, bok_id, kommentar
FROM bokningar
WHERE bokningar.typ='2'
and date_from<'".$today."'";
function that makes the foreach move ahead one day at the time...
function date_range($first, $last, $step = '+1 day', $output_format = 'Y-m-d' )
{
$dates = array();
$current = strtotime($first);
$last = strtotime($last);
while( $current <= $last ) {
$dates[] = date($output_format, $current);
$current = strtotime($step, $current);
}
return $dates;
}
foreach:
foreach (date_range($row['date_from'], $row['date_to'], "+1 day", "Y-m-d")
as $datespan)
if ($datespan < $today)
Query 2:
$query2 = "
SELECT bok_id, kommentar
FROM bokningar b
WHERE b.typ='2'
AND b.bok_id='".$row['bok_id']."'
AND b.weektype = '1'
AND b.".$dayForDate2." = '1'
AND NOT EXISTS
(SELECT t.tilldelad, t.bok_id
FROM tilldelade t
WHERE t.tilldelad = '".$datespan."'
AND t.bok_id='".$row['bok_id']."')
OR b.typ='2'
AND b.bok_id='".$row['bok_id']."'
AND b.weektype = '".$weektype2."'
AND b.".$dayForDate2." = '1'
AND NOT EXISTS
(SELECT t.tilldelad, t.bok_id
FROM tilldelade t
WHERE t.tilldelad = '".$datespan."'
AND t.bok_id='".$row['bok_id']."')";
b.weektype is either 1,2 or 3 (every week, every even week, every uneven week)
bokningar needs INDEX(typ, date_from)
Instead of computing $today, you can do
and date_from < CURDATE()
Are you running $query2 for each date? How many days is that? You may be able to build a table of dates, then JOIN it to bokningar to do all the SELECTs in a single SELECT.
When doing x AND y OR x AND z, first add parenthes to make it clear which comes first AND or OR: (x AND y) OR (x AND z). Then use a simple rule in Boolean arithmetic to transform it into a more efficient expression: x AND (y OR z) (where the parens are necessary).
The usual pattern for EXISTS is EXISTS ( SELECT 1 FROM ... ); there is no need to list columns.
If I am reading it correctly, the only difference is in testing b.weektype. So the WHERE can be simply
WHERE b.weektype IN ('".$weektype2."', '1')
AND ...
There is no need for OR, since it is effectively in IN().
tilldelade needs INDEX(tilldelad, bok_id), in either order. This should make the EXISTS(...) run faster.
Finally, bokningar needs INDEX(typ, bok_id, weektype) in any order.
That is a lot to change and test. See if you can get those things done. If it still does not run fast enough, start a new Question with the new code. Please include SHOW CREATE TABLE for both tables.
I want select X records from database (in PHP script), then sleep 60 seconds after continue the next 60 results...
SO:
SELECT * FROM TABLE WHERE A = 'B' LIMIT 60
SELECT SLEEP(60);
....
SELECT * FROM TABLE WHERE A = 'B' LIMIT X **where X is the next 60 results, then**
SELECT SLEEP(60);
AND etc...
How can I achievement this?
There is no such thing as "the next 60 records". SQL tables represent unordered sets. Without an order by, a SQL statement can return a result set in any order -- and even in different orders on different executions.
Hence, you first need something to guarantee the ordering . . . that is, an order by with keys that uniquely identify each row.
You can then use offset/limit to accomplish what you want. Or, you could put the code into a stored procedure and use a while loop. Or, you could do this on the application side.
In PHP:
<?php
// obtain the database connection, there's a heap of examples on the net, assuming you're using a library like mysqlite
$offset = 0;
while (true) {
if ($offset == 0) {
$res = $db->query('SELECT * FROM TABLE WHERE A = 'B' LIMIT 60');
} else {
$res = $db->query('SELECT * FROM TABLE WHERE A = 'B' LIMIT ' . $offset . ',60');
}
$rows = $db->fetch_assoc($res);
sleep(60);
if ($offset >= $some_arbitrary_number) {
break;
}
$offset += 60;
}
What you're doing is gradually incrementing the limit field by 60 until you reach a limit. The easiest way to do it is in a control while loop using true for the condition and break when you reach your invalid condition.
I'm just a beginner at mysql so in school we got task to do. It goes like this. Display / print 10% of all books from books in falling order. So i tried to use limit, but it doesn't work. What can i do? My code i've tried to use:
select title, price from book
order by price desc
limit (select count(*)*0.1 from book);
thank you for your answers!
limit values have to be hard-coded constants. You can't use variables on them, e.g. select ... limit #somevar is a syntax error. You also can't use sub-queries or other dynamic values either. So you're stuck with either fetching the row count ahead of time and stuff it into the query string as a "hard-coded" value:
$ten_percent = get_from_database('select count(*) / 10 from book');
$sql = "SELECT .... LIMIT $ten_percent";
Or you simply fetch everything and then abort your loop once you've reached 10%:
$sql = "SELECT ....";
$result = mysql_query($sql) or die(mysql_error());
$total_rows = mysql_num_rows($result);
$fetched = 0;
while($row = mysql_fetch_assoc()) {
$fetched++;
if ($fetched >= ($total_rows / 10)) {
break; // abort the loop at 10%
}
... do stuff with $row
}
I want to select currentrow as part of my query - I know I can loop over queries and get the currentrow variable, but I'm doing a QoQ before I use the rows and I want to keep the original rows, e.g.
//Original query
1, Audi
2, BMW
3, Skoda
//QoQ
1, Audi
3, Skoda
This is the code I've got:
q = new Query( datasource = application.db.comcar );
q.setSQL('
SELECT make, #rownum := #rownum +1 AS `rownumber`
FROM cars, ( SELECT #rownum :=0 )
LIMIT 10
');
r = q.execute().getResult();
But it's throwing the following error:
Parameter '=' not found in the list of parameters specified
SQL: SELECT make, #rownum := #rownum + 1 AS `rownumber` FROM cars, ( SELECT #rownum :=0 ) LIMIT 10
This will work in cfquery but I'd like to use it in CFScript. Is there an alternative to using := or some way of escaping this in the query.
It looks like this is a bug in Coldfusion. I could change my code to use cfquery but I'd rather not mix script and tags in my page.
So my workaround was is as follows:
/*
* based on the existing query 'tmpFields'
*/
// build array of row numbers
arrRowNumbers = [];
cntRowNumbers = tmpFields.recordCount;
for( r = 1; r <= cntRowNumbers; r++ ) {
arrayAppend( arrRowNumbers, r );
}
// add a new column with the new row number array
queryAddColumn( tmpFields, "fieldNumber", "integer", arrRowNumbers );