I need to show some records sorted based on modified column (latest activity on top)
(Post with new edit or comments at the top)
App UI has twitter like 'more' post button for infinite scroll. each 'more' will add next 10 records to UI.
Issue is that pagination index breaks when any of the to be shown record is modified
for example
Suppose i have records A,B,C,..Z in jobs table.
first time I'm' showing the records A-J to the user using
SELECT * FROM Jobs WHERE 1 ORDER BY last_modified DESC LIMIT 0, 10
second time if none of the records are modified
SELECT * FROM Jobs WHERE 1 ORDER BY last_modified DESC LIMIT 10, 10
will return K-T
But if some body modifies any records after J before the user clicks 'more button',
SELECT * FROM Jobs WHERE 1 ORDER BY last_modified DESC LIMIT 10, 10
will return J-S
Here record J is duplicated. I can hide it by not inserting J to the UI, but the more button will show only 9 records. But this mechanism fails when large number of records are updated, If 10 records are modified, the query will return A-J again.
What is the best way to handle this pagination issue?
Keeping a second time stamp fails if a record has multiple updates.
Server cache of queries?
I would do a NOT IN() and a LIMIT instead of just a straight LIMIT with a pre-set offset.
SELECT * FROM Jobs WHERE name NOT IN('A','B','C','D','E','F','G','H','I','J')
ORDER BY last_modified DESC LIMIT 10
This way you still get the most recent 10 every time but you would need to be tracking what IDs have already been shown and constantly negative match on those in your sql query.
Twitter timelines not paged queries they are queried by ids
This page will help you a lot understanding timeline basics https://dev.twitter.com/docs/working-with-timelines
lets say each column have id field too
id msg
1 A
2 B
....
First query will give you 10 post and max post_id will be 10
Next query should be
SELECT * FROM Jobs WHERE id > 10 ORDER BY last_modified DESC LIMIT 0, 10
I don't know the exact solution but I can give it a try.
First u need an integer ID column in your Job table.
Now send a max_id = null along with limit = 10 and offset = 0 from UI.
In this case if max_id is null, set max_id to (MAX(ID) + 1) of Table.
SELECT (MAX(ID) + 1) INTO max_id FROM Jobs;
Later find the records:
SELECT * FROM Jobs WHERE ID < max_id ORDER BY last_modified DESC LIMIT 10 OFFSET 0;
Return the records to UI.
Now from UI set max_id = ID of first record in the response array, offset = offset + limit.
Now onwards try with updated values of max_id and offset:
SELECT * FROM Jobs WHERE ID < max_id ORDER BY last_modified DESC LIMIT 10 OFFSET 10;
Related
I have one table e.g. Employee, which has columns (id, name, is_active).
I want to fetch the records from employee table sorted by name along with pagination.
SELECT * from employee ORDER BY name ASC LIMIT 5, 10;
In above query 5 count to skip first 5 records and take next 10 records.
I want to make a query which will return all records which has value is_active=1, followed by records which has value is_active = 0 with pagination.
e.g: I have 50 records out of them 40 records has value is_active=1, and 10 records which has value is_active=0.
So as I am using pagination and page size is suppose 10 records per page.
So query will return all active records first with order by name and at last page will return 10 inactive records order by name.
Does anyone have any suggestions? Any help would be greatly appreciated.
Thank you!
You just need descendingly sort by is_active column, and no need an OFFSET value depending on the explanation such as
SELECT *
FROM employee
ORDER BY is_active DESC, name
LIMIT 10
I have a MySQL Database with over 100k rows, so I need to make a search to fetch only the last 1000 rows , so if it is not found in the last 1000 rows the fetch ends (even if it is not found)
Example: if my table is like that
id name
1 AL
2 BL
...
1000 P12
1001 P15
And I do a fetch like this: SELECT * FROM myTable WHERE name = 'AL' ONLY LAST 1000 ROWS ORDER BY id DESC (Since I don't know what to use I invented the ONLY LAST 1000 ROWS)
This should return empty because I wanted my query to get the information only if it was on the last 1000 rows, not on the 1001th as specified.
Using LIMIT field doesn't work as it would LIMIT the FOUND ROWS not when they are not found.
Is there a way to implement this in MySQL ?
Thank you!
As touched on in the comments, you can use OFFSET to get the id of the 1000th last record, then SELECT records with an id larger than that record's id.
Something like this:
SELECT name
FROM myTable
WHERE id > (SELECT id FROM myTable ORDER BY id DESC LIMIT 1 OFFSET 1000)
AND name = 'AL'
I have hundreds of thousands of price points spanning 40 years plus. I would like to construct a query that will only return 3000 total data points, with the last 500 being the most recent data points, and the other 2500 being just a sample of the rest of the data, evenly distributed.
Is it possible to do this in one query? How would I select just a sample of the large amount of data? This is a small example of what I mean for getting just a sample of the other 2500 data points:
1
2
3
4
5
6
7
8
9
10
And I want to return something like this:
1
5
10
Here's the query for the last 500:
SELECT * FROM price ORDER BY time_for DESC LIMIT 500
I'm not sure how to go about getting the sample data from the other data points.
Try this:
(SELECT * FROM price ORDER BY time_for DESC LIMIT 500)
UNION ALL
(SELECT * FROM price WHERE time_for < (SELECT time_for FROM price ORDER BY time_for LIMIT 500, 1) ORDER BY rand() LIMIT 2500)
ORDER BY time_for
Note: It's probably going to be slow. How big is this table?
It might be faster to only get the primary ID from all these rows, then join it to the original in a secondary query once it's narrowed down. This is because ORDER BY rand() LIMIT has to sort the entire table. If the table is large this can take a LONG time, and a lot of disk space. Retrieving only the ID reduces the necessary disk space.
The previous answer is good, but you did specify that you want the results to be evenly distributed so I'll add this possibility too. By iterating a counter over the rows you can use a MOD operator to sample an even distribution. I don't have a MYSQL install right now to test this so apologies if the syntax isn't 100% spot on. But it should be close enough and may give you some ideas.
( SELECT p1.*
FROM price p1
ORDER BY p1.time_for DESC
LIMIT 500 )
UNION ALL
( SELECT #i := #i + 1 AS row_num,
p2.*
FROM price p2,
(SELECT #i: = 0)
WHERE row_num > 500
AND (row_num % 500) = 0
ORDER BY time_for DESC )
The first query gives the 500 latest rows. The second query gives every 500th row after that, thus returning an even distribution from the rest of the data. Obviously you can tune this parameter to achieve the desired sample spacing. Or base it on the total number of rows in the table to calculate the necessary spacing to give exactly 2500 records.
I have a table and I try to retrieve a certain number of random entries and I want to put a certain maximum number of entries of a category at the top, followed by a certain number of another category.
If there is no entries in the first category, the allowed number of entries of it would be added to the number of entries of the 2sd category.
For example, my table has 3 categories (category: 'complete', 'partial', 'empty'). I want to display 12 entries in total.
For the "complete" category, I want to display 8. Then display 3 partials and finally 1 empty.
If there is less than 8 complete, let say there is 5, then it would display 5 complete, 6 partials and 1 empty. (missing 3 complete, so 3 base partials + 3 extra one = 6).
What would be the query to do that?
Here is the query I actually have:
SELECT *
FROM items
ORDER BY category='complete' DESC, category='partial' DESC, category DESC, RAND() ASC
LIMIT 12
The problem with the above query is that it gets 12 completes and it isn't exactly what I am looking for.
You can use a USION DISTINCT statement.
Try like this :
(SELECT * FROM items WHERE category = "complete" LIMIT 8)
UNION DISTINCT (SELECT * FROM items WHERE category = "partial" LIMIT 3)
UNION DISTINCT (SELECT * FROM items WHERE category = "" LIMIT 1)
I have only one table with the name of offers and it has multiple offers in it like each time we pull in an offer, we create a new row for example: for travelling to Timbuktu, there can be 10 or more rows each containing an offer, each time a offers comes in, it is being saved with PHP unix timestamp in the column name 'created_on', so to figure out which offer is latest, I am currently using following query:
SELECT * FROM offers WHERE city= 'Timbuktu' AND created_on=(SELECT max(created_on)from offers WHERE city = 'Timbuktu')
This serves the purpose if I have to fetch only one latest row, if say I want to fetch last 4 or 8 rows with the greatest timestamp, how I can do that in most efficient way?
SELECT *
FROM offers
WHERE city= 'Timbuktu'
order by created_on desc
limit 0, 8
and for 1 row you can use same request just replace 8 with 1
SELECT * FROM offers WHERE city='Timbuktu' ORDER BY created_on DESC LIMIT 4;