I'm trying to block certain IP addresses and IP ranges from performing certain actions. Right now, I SELECT * FROM blocked WHERE ip=$_SERVER['REMOTE_ADDR'] and if ->num_rows returns > 0, I redirect the user.
This works with full IP addresses but if I put 199.90.*.* for example, into the database, how might I match that? Also, what would be the best way to store the IPs in the database?
The answer to your question is to use like:
SELECT *
FROM blocked
WHERE $_SERVER['REMOTE_ADDR'] like replace(ip, '*', '%')
The wildcard in like is '%' rather than '*', so this replaces the asterisk with the percent sign. I am guessing that you would actually store the string like '199.199.%.%', but the above is for clarity.
Although this technically solves your problem, it has other issues because MySQL may not use an index for these comparisons (see here). The impact depends on how large the blocked table is. If it has 100 rows, this may not be an issue. If it has 100,000 then it probably is.
An alternative to using wildcards for the blocked table is to have FromIP and ToIP as columns. Then something like "199.199.." would simply be stored as "199.199.000.000" and "199.199.999.999". The query would be:
where $_SERVER['REMOTE_ADDR'] between FromIP and ToIP
And you would have an index on blocked(FromIP, ToIP).
I would first check for full IP address, then mask out the last byte with *, and check for that, then the last 2 bytes, and so on. This way you go from specific to less specific, and if once you get a hit, the IP is blocked. It's probably not the most efficient way, but I guess you have to do it only once, at login-time.
I would store IP addresses for simplicity as strings.
Another solution could be to store IP address as byte-array, but that wouldn't make anything easier I guess, and would be much less maintainable with for example the command-line tool SQL tool.
Your best solution would be to use INET_ATON function.
Given the dotted-quad representation of an IPv4 network address as a string, returns an integer that represents the numeric value of the address in network byte order (big endian).
So, if you want to look for an IP in a certain range, you would use a Query like this:
SELECT * FROM blocked WHERE INET_ATON($_SERVER['REMOTE_ADDR']) between INET_ATON(this_ip) and INET_ATON(this_other_ip).
As I think, the best way to store IPv4 addresses in a MySQL Data base is using that function, to convert them to an int value and store it like that. And If you want to get the IP from it's int representation use INET_NTOA.
Storing IP's as int, the query ends like this:
SELECT * FROM blocked WHERE INET_ATON($_SERVER['REMOTE_ADDR']) between this_ip and this_other_ip.
And instead of using 199.90.*.* use (199.90.0.0).
Related
I have issues finding the right approach to properly return (and match) the first /64 bits of IPv6 addresses stored in the DB like: 2a02:a420:0003:7dbc:0002:0001:b693:9622
I need to retrieve entries in the table that are using the same network/subnet of IPv6 (/64 bits)
note: the table includes both IPv4 and IPv6 addresses
i am using MySQL Workbench 8.0
Let's say you have addrcolumn stored as a VARCHAR(39) text string with IPv6 addresses, and you have #matchaddr as a text string containing a valid address within the /64 range you want to match.
You can get the first 16 hex digits of addrcolumn like this:
SUBSTRING(HEX(INET6_ATON(addrcolumn)),1,16)
So you can do your match with the LIKE operator like this.
WHERE HEX(INET6_ATON(addrcolumn))
LIKE CONCAT(SUBSTRING(HEX(INET6_ATON(#matchaddr)),1,16), '%')
After the string-manipulation functions this comes out looking like
WHERE `2a02a42000037dbc00020001b693962a' LIKE '2a02a42000037dbc%'
You need the mischegoss with the INET6_ATON() because IPv6 addresses are often shortened with two colons meaning a bunch of zeros like this:
2a02:a420::2:1:b693:9622
INET6_ATON() copes with that correctly.
Edit But, if you assume your representation of ipv6 addresses never, ever, contains any shortening via : or :1:2: style abbreviation, you can just do substring matching with LIKE.
WHERE addrcolumn LIKE CONCAT(SUBSTRING(#matchaddr)),1,19), '%')
Note the 19-character substring. This comes out looking like
WHERE `2a02:a420:0003:7dbc:0002:0001:b693:962a'
LIKE '2a02:a420:0003:7dbc%'
Beware: real users will violate your assumption.
In MySQL 8+ you can turn a binary representation into an unabbreviated text representation like this:
SELECT LOWER(
SUBSTRING(
REGEXP_REPLACE(HEX(binaryAddr), '([0-9a-f]{4})', '$1:'), 1, 39))
In MariaDB 10+ it is:
SELECT LOWER(
SUBSTRING(
REGEXP_REPLACE(HEX(binaryAddr), '([0-9a-f]{4})', '\\1:'), 1, 39))
Will the MariaDB or MySQL team add some IPv6-specific indexing / filtering stuff one day? It would be very nice if they did.
I am working in a web apps, I made some checking/limitation when user trying to login my application. Now i need to make a white list function to pass these checking make use of user ip address.
I'm trying to write a sql statement to get matched IP address in order to achieve white list. If the sql return data then pass the checking , if not just continue checking.
However, whitelist table in database need to be support 192.168.* or 192.* or (*. *.1.1) .So it will return data and pass if the user ip is 192.X.X.X
SELECT * FROM whitelist WHERE ip_address = $ip;
my sql statement like this.
I agree with #arno comment. If you have limited value to check then use regex instead of database call. It will save you time.
But if you want to call database then I remember that MySql support regex in query also
SELECT
*
FROM
whitelist
WHERE
ip_address REGEXP '^((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$';
Above regex is to check all valid IP address. But you can change it based on your requirement.
There is no natural case when you need to match something like *.*.1.1.
IP addresses are mostly matched according to subnets in their CIDR notation. Because this is how networks are organized.
Even simpler, you can convert IP addresses to a long datatype using INET_ATON() and make simple matches using > and <.
Please refer to these solutions:
Is there way to match IP with IP+CIDR straight from SELECT query?
https://dba.stackexchange.com/questions/171044/determining-if-an-ip-is-within-an-ipv4-cidr-block
You can use the LIKE statment:
SELECT * FROM whitelist WHERE ip_address LIKE '192.168.%';
The % character allows any character to replace it, therefore the above query will return IP adress in the range 192.168.0.0 - 192.168.255.255, provided you indeed only have IP adresses in this field.
I have installed the Maxmind GeoIP database and now I am testing the results. I live in Amsterdam, so I did a IP query check with my own IP address, but I got 2 results back. Am I doing something wrong or is the data not clear?
Database example:
SQL query:
SELECT * FROM wp_geoip WHERE '{my-ip-address}' BETWEEN begin_ip_num AND end_ip_num;
Results:
You can't just store IP addresses as a varchar. Well... you can, but it's wrong on something of a fundamental level.
The correct solution is to store the IP addresses as what they actually represent: unsigned 32 bit integers (INT UNSIGNED).
Convert the data, when you import it, using the INET_ATON() built-in function, which converts a dotted-quad IPv4 address into an unsigned integer.
Query the data using the inverse function:
WHERE INET_NTOA('you.r.ip.add') BETWEEN begin_ip_num AND end_ip_num;
You will get better performanace if you index the begin and end columns in both directions, e.g.:
PRIMARY KEY(begin_ip_num,end_ip_num),
KEY(end_ip_num,begin_ip_num)
However... B-Trees are not optimal for this kind of search.
You will also be able to query it even faster, still, if you use a spatial index, as Jeremy Cole explains in a blog post on the topic. Note that he also goes into detail about the use of INET_ATON() and INET_NTOA().
The spatial index concept blows some people's minds, since they assume "spatial" means only "geospatial" but IP address space is, after all, still "space" and R-Tree indexes provided by MySQL's spatial extensions are much more optimal than B-Trees for searching for the boundaries of a "space" that a thing (like an IP address) occupies.
It probably uses the begin_ip_num and end_ip_num as a varchar-field.
So an IP that starts with 92.11[...] will also be between the range in the first record. For the first records it checks for any string between 92.0 and 92.3
So doing a text search will get you wrong results in this case. You could try to convert all IPs to a good searchable string. That means an ip like 92.31.255.255 should be converted to 092.031.255.255
If you do that for all you IPs you can do a proper search on them.
CREATE FUNCTION dbo.formatIP ( #ip varchar(20) )
RETURNS varchar(20)
AS
BEGIN
SELECT RIGHT('000' + PARSENAME(#ip,4), 3) + '.' +
RIGHT('000' + PARSENAME(#ip,3), 3) + '.' +
RIGHT('000' + PARSENAME(#ip,2), 3) + '.' +
RIGHT('000' + PARSENAME(#ip,1), 3)
END
Wrap that up in a function and use it like this:
SELECT * FROM wp_geoip
WHERE dbo.formatIP('{my-ip-address}')
BETWEEN dbo.formatIP(begin_ip_num) AND dbo.formatIP(end_ip_num);
I'm using sqlite3 to try and find users who have an e-mail address that is either with Gmail, Yahoo or Hotmail. It needs to do this just on the basis of the first part of the domain, so I want any address that had the #yahoo to be accepted.
It appears from documentation that it is not possible to use a regular expression when querying an sqlite database. Is there any elegant way of doing something similar? It doesn't seem to be possible to use a "like/in" with multiple options (eg: LIKE (%#yahoo%, %#gmail%, %#hotmail%)?
Failing that, I may switch over to MySQL for a reg exp as I want to keep the solution simple and elegant and DB isn't a major factor. How would said regexp query be written in MySQL?
You can't use multiple "LIKE" in that way but you can use:
(email LIKE "%#yahoo%" OR email LIKE "%#gmail%" OR ....)
you can use the way Nemoden is using or something like this (not tested)
WHERE email REGEXP '[\w\.]+#(yahoo|gmail|ymail|hotmail)\.(com|ru|co\.uk)'
This would be much faster because LIKE %string% OR ... is very slow and doesnt use any indexes(dont know if REGEXP uses indexes tho).
I think you might want something like this (RLIKE function):
WHERE email RLIKE '^[A-Z0-9._%+-]+#[A-Z0-9.-]+\.([A-Z]{2}|com|org|net|edu|gov|mil|biz|info|mobi|name|aero|asia|jobs|museum|travel)$'
If you use whatever_cs (case sensitive collation), use a-zA-Z instead of A-Z.
You can also get rid of ^ and $ in the regex. ^ means "starts with" and $ means "ends with"
UPDATE:
'.*#(gmail|hotmail|yahoo)\.[A-Z]{2,4}'
Assume a table titled 'transactions'
Typical query: select * from transactions where ip=INET_ATON('127.0.0.1');
I want the ability to do a regex search for all ip's with a particular first octet (i.e. 127) but I can't quite figure out the syntax.
Thanks!
‘transactions.ip’ is already an integer, which is why the questioner is using INT_ATON. It's no good trying to match it directly against a string like/regexp, you would have to convert it back to an IPv4 address first using the reverse function INT_NTOA:
SELECT * FROM transactions WHERE INT_NTOA(ip) LIKE '127.%';
This requires a complete table scan (defeating indexes) to convert and compare.
I want the ability to do a regex search for all ip's with a particular first octet (i.e. 127)
However, in the case where you want to compare against the front part of an IP address, you can take advantage of the fact that the first octet is stored in the most significant bits of the number, and say:
SELECT * FROM transactions WHERE ip BETWEEN INT_NTOA('127.0.0.0') AND INT_NTOA('127.255.255.255');
Which can be done using an index on ‘ip’ so could be considerably more efficient.
You are comparing ip to the result of INET_ATON, which implies that ip is a 32-bit integer. Your operation should be simply...
SELECT * FROM transactions WHERE (longip & 0xFF000000) = (0x7F000000)
This will be far faster than applying a regex to a dotted IP in a string, and faster than the LIKE '127.%' solution.
Like this?
select * from transactions where ip REGEXP "^[1-2][1-9][1-9]"
SELECT * FROM transactions WHERE ip LIKE '127.%';