Filtering SQL results - mysql

I would like to show a filtered result to a few ip's that keep scraping my content. I have blocked them with .htaccess and they change their ip address and continue doing it. So I thought, I want to create a soft block that won't show them all of my content and hopefully they won't even notice.
My table has a auto_increment field
id | category | everything else
1 1
2 1
3 4
4 2
I have been trying something like this.
SELECT * from mytable WHERE `category` = '1' having avg(id/3) = 1 ORDER BY `id` DESC LIMIT 0 , 10
I have searched forever but I am a newb to sql, so I don't even really know what I am searching for. I hope somebody here can please help me! Thanks :)

If you want to get remainder of division by 3, you should use % operator.
SELECT * from mytable WHERE `category` = '1' and id % 3 = 1 ORDER BY `id`
DESC LIMIT 0 , 10

Generally, the ID column is not for doing computations on it. It does not represent anything other than unique identifier of the record (at most, it should be used to sort the records chronologically) - you could have there GUIDs for example, and your application should work.
If you want to store the IPs that you want to block in your DB, consider adding another column to your table, call it status or something similar, and store in this column the status for that ip - this status could be clean, suspicious, blocked, etc. After that, your SELECT should look only after the rows with blocked status

Related

MYSQL pagination performance

I have the following sample MYSQL table:
id | count_likes
-----------
1 | 30
2 | 95
3 | 60
4 | 60
5 | 22
I want to order the table by column count_likes descending and display 5 rows at a time (this is a sample table so assume thousands of rows).
To achieve this I run the following command:
SELECT * FROM table ORDER BY count_likes DESC, id DESC LIMIT 5
I want to give the option for users to load more rows like loading facebook comments for example (5 rows at a time).
To achieve this I run the following command:
SELECT * FROM table WHERE id NOT IN(values already loaded)
ORDER BY count_likes DESC, id DESC LIMIT 5
This could work well for few pages but I think it's not recommended to have like hundred values in the WHERE NOT IN clause.
If I make the command like this:
SELECT * FROM table WHERE count_likes < 'the last displayed count number'
I could miss some rows which have the same count like the last loaded row.
If I make the command like this:
SELECT * FROM table WHERE count_likes <= 'the last displayed count number'
I could get duplicate values that are already loaded.
If I make the command like this:
SELECT * FROM table ORDER BY count_likes DESC LIMIT offset,5
I may get disorganized or duplicate rows as the count_likes for any row may increase or decrease while other users are manipulating the same page.
What is the best way to load more rows in my case above?
The most accurate one would be the WHERE NOT IN but I don't know if it causes performance issues on large number of members like hundred or even thousand.

SQL query to get all except where id is in array that comes from another table

I could use some help with some SQL. Part of a website that I need to do is a place where you can add/remove stores from users. Now the query to get the stores that have already been assigned to a certain user is simple, however I can't seem to figure out how to do the opposite. Meaning getting all that haven't been added to the user yet.
So here's a template of 1 of the tables.
user_id store_id
1 11
1 12
1 14
2 15
4 16
If I run this (that 1 will be php variable): SELECT store_id FROM store_user WHERE user_id= 1;
I get the result like that:
store_id
11
12
14
And the rest would be something like: SELECT * FROM store except where id = 11,12,14.
I would appreciate if anyone could help me with that last part and preferrably put it all in to 1 query.
Edit: for those who stumble here with a similar problem, this is what worked in the end:
SELECT * FROM store where id NOT IN (select store_id from store_user where user_id = 1)
Thanks for the help everyone!
You can use IN or NOT IN
select store_id from store where id in (1,2....)
select store_id from store where id not in (1,2....)
use NOT IN
SELECT * FROM store where
id NOT IN (select store_id from store_user where user_id = 1)

How to link slightly differing times

I have two log tables that I would like to link, but the entries made in each table are not done at exactly the same time and the time difference varies but should always be within a second.
To keep it simple, let's say table A looks like:
ItemId int
Comment varchar(50)
LogTime datetime
and let's say that table B has the exact same structure.
Suppose these records are in Table A:
ItemId Comment LogTime
-----------------------------------
100 Test100-A1 12:00:00.00
200 Test200-A 12:00:03.50
100 Test100-A2 12:00:06.30
and these are in Table B
ItemId Comment LogTime
-----------------------------------
100 Test100-B1 12:00:00.03
200 Test200-B 12:00:02.98
100 Test100-B2 12:00:06.53
And I'd like to have the following output
A.ItemId A.Comment A.LogTime B.ItemId B.Comment B.LogTime
-------------------------------------------------------------------------
100 Test100-A1 12:00:00.00 100 Test100-B1 12:00:00.03
200 Test200-A 12:00:03.50 200 Test200-B 12:00:02.98
100 Test100-A2 12:00:06.30 100 Test100-B2 12:00:06.53
How can I create a query that will link the two tables together this way on the ItemId and LogTime, but with up to a 1 second variation in either direction for the LogTime?
I figured it out... was actually a bit simpler than I realized.
select *
from A left join
B on A.ItemId = B.ItemId and
abs(DATEDIFF(ss, A.LogTime, B.LogTime)) <= 1
I tried doing it based on milliseconds instead of seconds the first time but that was giving me an overflow error when comparing dates that were too far apart. I'd rather do milliseconds though so I can narrow it down to a little less than a second but not sure what the best way to accomplish that just yet. Maybe I could use a case statement. If someone else wants to post an answer that does it I'll mark it or else I'll come back later and update my answer to work off milliseconds when I get a chance.

Complex MySQL COUNT query

Evening folks,
I have a complex MySQL COUNT query I am trying to perform and am looking for the best way to do it.
In our system, we have References. Each Reference can have many (or no) Income Sources, each of which can be validated or not (status). We have a Reference table and an Income table - each row in the Income table points back to Reference with reference_id
On our 'Awaiting' page (the screen that shows each Income that is yet to be validated), we show it grouped by Reference. So you may, for example, see Mr John Smith has 3 Income Sources.
We want it to show something like "2 of 3 Validated" beside each row
My problem is writing the query that figures this out!
What I have been trying to do is this, using a combination of PHP and MySQL to bridge the gap where SQL (or my knowledge) falls short:
First, select a COUNT of the number of incomes associated with each reference:
SELECT `reference_id`, COUNT(status) AS status_count
FROM (`income`)
WHERE `income`.`status` = 0
GROUP BY `reference_id`
Next, having used PHP to generate a WHERE IN clause, proceed to COUNT the number of confirmed references from these:
SELECT `reference_id`, COUNT(status) AS status_count
FROM (`income`)
WHERE `reference_id` IN ('8469', '78969', '126613', ..... etc
AND status = 1
GROUP BY `reference_id`
However this doesn't work. It returns 0 rows.
Any way to achieve what I'm after?
Thanks!
In MySQL, you can SUM() on a boolean expression to get a count of the rows where that expression is true. You can do this because MySQL treats true as the integer 1 and false as the integer 0.
SELECT `reference_id`,
SUM(`status` = 1) AS `validated_count`,
COUNT(*) AS `total_count`
FROM `income`
GROUP BY `reference_id`

MySql order by specific ID values

Is it possible to sort in MySQL by "order by" using a predefined set of column values (ID) like order by (ID=1,5,4,3) so I would get records 1, 5, 4, 3 in that order out?
UPDATE: Why I need this...
I want my records to change sort randomly every 5 minutes. I have a cron task to update the table to put different, random sort order in it.
There is just one problem! PAGINATION.
I will have visitors who come to my page, and I will give them the first 20 results. They will wait 6 minutes, go to page 2 and have the wrong results as the sort order has already changed.
So I thought that if I put all the IDs into a session on page 2, we get the correct records even if the sorting had already changed.
Is there any other better way to do this?
You can use ORDER BY and FIELD function.
See http://lists.mysql.com/mysql/209784
SELECT * FROM table ORDER BY FIELD(ID,1,5,4,3)
It uses Field() function, Which "Returns the index (position) of str in the str1, str2, str3, ... list. Returns 0 if str is not found" according to the documentation. So actually you sort the result set by the return value of this function which is the index of the field value in the given set.
You should be able to use CASE for this:
ORDER BY CASE id
WHEN 1 THEN 1
WHEN 5 THEN 2
WHEN 4 THEN 3
WHEN 3 THEN 4
ELSE 5
END
On the official documentation for mysql about ORDER BY, someone has posted that you can use FIELD for this matter, like this:
SELECT * FROM table ORDER BY FIELD(id,1,5,4,3)
This is untested code that in theory should work.
SELECT * FROM table ORDER BY id='8' DESC, id='5' DESC, id='4' DESC, id='3' DESC
If I had 10 registries for example, this way the ID 1, 5, 4 and 3 will appears first, the others registries will appears next.
Normal exibition
1
2
3
4
5
6
7
8
9
10
With this way
8
5
4
3
1
2
6
7
9
10
There's another way to solve this. Add a separate table, something like this:
CREATE TABLE `new_order` (
`my_order` BIGINT(20) UNSIGNED NOT NULL,
`my_number` BIGINT(20) NOT NULL,
PRIMARY KEY (`my_order`),
UNIQUE KEY `my_number` (`my_number`)
) ENGINE=INNODB;
This table will now be used to define your own order mechanism.
Add your values in there:
my_order | my_number
---------+----------
1 | 1
2 | 5
3 | 4
4 | 3
...and then modify your SQL statement while joining this new table.
SELECT *
FROM your_table AS T1
INNER JOIN new_order AS T2 on T1.id = T2.my_number
WHERE ....whatever...
ORDER BY T2.my_order;
This solution is slightly more complex than other solutions, but using this you don't have to change your SELECT-statement whenever your order criteriums change - just change the data in the order table.
If you need to order a single id first in the result, use the id.
select id,name
from products
order by case when id=5 then -1 else id end
If you need to start with a sequence of multiple ids, specify a collection, similar to what you would use with an IN statement.
select id,name
from products
order by case when id in (30,20,10) then -1 else id end,id
If you want to order a single id last in the result, use the order by the case. (Eg: you want "other" option in last and all city list show in alphabetical order.)
select id,city
from city
order by case
when id = 2 then city else -1
end, city ASC
If i had 5 city for example, i want to show the city in alphabetical order with "other" option display last in the dropdown then we can use this query.
see example other are showing in my table at second id(id:2) so i am using "when id = 2" in above query.
record in DB table:
Bangalore - id:1
Other - id:2
Mumbai - id:3
Pune - id:4
Ambala - id:5
my output:
Ambala
Bangalore
Mumbai
Pune
Other
SELECT * FROM TABLE ORDER BY (columnname,1,2) ASC OR DESC