Supply user host pairs to be excluded from mysql results - mysql

Data to work with:
+-------------+-------------+
| user | host |
+-------------+-------------+
| user1 | host1 | -
| user1 | ip1 | -
| user1 | host2 | *
| user2 | host2 | -
| user2 | ip2 | -
| unknown | unknown | +
| user1 | unknown | +
| unknown | host | +
+-------------+-------------+
The symbol to the right of the table are:
- do not show | + show as they are unknown | * because a user can only connect to one host, unless I had authorised it in which case I would supply the user host pair to the call also and it would not show.
Thats how I want things to work anyway.
This is where I am at with help from my question and as there is now a further condition a new question needs asking.
Current procedure
USE mysql;
DROP PROCEDURE IF EXISTS ShowUsers;
DELIMITER $
CREATE PROCEDURE `ShowUsers`(
IN KnownUsers varchar(500),
IN KnownHosts varchar(500)
)
BEGIN
SELECT
user,host
FROM
user
WHERE
NOT FIND_IN_SET(host, KnownHosts)
AND
NOT FIND_IN_SET(user, KnownUsers)
ORDER BY user, host ASC;
END $
DELIMITER ;
Calling the procedure like this
# known users and known hostnames or ips to match and exclude from results.
SET #Usernames = 'user1,user2';
SET #Hostnames = 'host1,host2,ip1,ip2'
CALL ShowUsers(#Usernames, #Hostnames);
Intended Result:
+-------------+-------------+
| user | host |
+-------------+-------------+
| user1 | host2 | *
| unknown | unknown | +
| user1 | unknown | +
| unknown | host | +
+-------------+-------------+
I want to be able to supply multiple user:host pairs of (known legitimate credentials) and return results that do not match, so return only suspect/illegitimate credentials in the query results.
I have created a fiddle https://www.db-fiddle.com/f/xb7dWXbkokHGbcPdzR7BUa/4 Hopefully you can see where I am going with this.

Based on whatever I could understand from your problem statement, you will need to use multiple string operations to satisfy your conditions (explanation in inline comments below):
Query
SELECT
`user`,`host`
FROM
tbl
WHERE
-- NOT condition to avoid returning one-to-one mapping between `user` and `host`
-- If `user` exist in the #Usernames, and the position of the
-- `user` matches with the position of the `host` in the #Hostnames
NOT (
FIND_IN_SET(`user`, #Usernames) > 0
-- Host and User are at same position in the lists
AND FIND_IN_SET(`user`, #Usernames) = FIND_IN_SET(`host`, #Hostnames)
)
AND
-- NOT condition to handle `host` at the end of #Hostnames list, where
-- there is no corresponding `user` mapped
NOT (
FIND_IN_SET(`host`, #Hostnames) > CHAR_LENGTH(#Usernames)
- CHAR_LENGTH(REPLACE(#Usernames, ',', ''))
+ 1
);
Result
| user | host |
| ------- | ------- |
| user1 | host2 |
| unknown | unknown |
| user1 | unknown |
| unknown | host |
View on DB Fiddle
Caveat: Above query will not work when there is no user in the #Usernames list. For brevity purpose, I avoided making the conditions more complex to handle that. Moreover, I doubt that in your practical use-case, you would have a situation where there are no user in the list.

This construct works (but not efficiently for huge tables):
WHERE (user, host) NOT IN ( ('u1', 'h1'), ('u2', 'h2), ... )
For further discussion, see "row constructor".

Related

Freeradius 3.0 doesn't send any data through mysql (radacct table empty)

I have deployed a freeradius server version 3.0 with MySQL in Ubuntu (20.04).
There are the respective versions of the radius and MySQL:
root#server:/etc/freeradius/3.0# mysql -V
mysql Ver 8.0.23-0ubuntu0.20.04.1 for Linux on x86_64 ((Ubuntu))
root#server:/etc/freeradius/3.0# freeradius -v
radiusd: FreeRADIUS Version 3.0.20, for host x86_64-pc-linux-gnu, built on Jan 25 2020 at 06:11:13
FreeRADIUS Version 3.0.20
Copyright (C) 1999-2019 The FreeRADIUS server project and contributors
My situation is: from a mobile phone I need to connect to an access point and authenticate to that WLAN network (802.11i-WPA-802.1x) using a username and password specified in the radius database.
I would like to limit the number of concurrent logins per user (demouser in this case) to 1 because i only have one unique user. I have spent a lot of time searching the forums and documentation, but can't find anything to figure it.
This is the output of freeradius in debug mode.
I have several doubts about what would be the best way to create the user in the database.
At the moment it is configured in a basic way, that is to say I have specified in the radcheck table the user and in the nas table my client (the access point):
mysql> SELECT * FROM nas;
+----+-------------+-----------+-------+-------+----------+--------+-----------+---------------+
| id | nasname | shortname | type | ports | secret | server | community | description |
+----+-------------+-----------+-------+-------+----------+--------+-----------+---------------+
| 1 | 172.20.1.20 | AP_1 | other | NULL | 12345678 | NULL | NULL | Client Radius |
+----+-------------+-----------+-------+-------+----------+--------+-----------+---------------+
1 row in set (0.00 sec)
mysql> SELECT * FROM radcheck; SELECT * FROM radgroupcheck; SELECT * FROM radgroupreply; SELECT * FROM radusergroup;
+----+----------+--------------------+----+----------+
| id | username | attribute | op | value |
+----+----------+--------------------+----+----------+
| 1 | demouser | Cleartext-Password | := | demopass |
+----+----------+--------------------+----+----------+
1 row in set (0.00 sec)
+----+------------+------------------+----+-------+
| id | groupname | attribute | op | value |
+----+------------+------------------+----+-------+
| 1 | demo_group | Simultaneous-Use | := | 1 |
+----+------------+------------------+----+-------+
1 row in set (0.00 sec)
+----+------------+--------------------+----+---------------------+
| id | groupname | attribute | op | value |
+----+------------+--------------------+----+---------------------+
| 1 | demo_group | Service-Type | := | Framed-User |
| 2 | demo_group | Framed-Protocol | := | PPP |
| 3 | demo_group | Framed-Compression | := | Van-Jacobsen-TCP-IP |
+----+------------+--------------------+----+---------------------+
3 rows in set (0.00 sec)
+----+----------+------------+----------+
| id | username | groupname | priority |
+----+----------+------------+----------+
| 1 | demouser | demo_group | 0 |
+----+----------+------------+----------+
1 row in set (0.00 sec)
And if I see the logs file the authentications makes successfull:
This is the output of /var/log/freeradius/sqllog.sql
INSERT INTO radpostauth (username, pass, reply, authdate) VALUES ( 'demouser', '', 'Access-Accept', '2021-04-27 09:04:19');
This is the other output of /var/radacct/172.20.1.20/auth-detail....
Tue Apr 27 09:37:46 2021
Packet-Type = Access-Request
User-Name = "demouser"
Service-Type = Framed-User
NAS-IP-Address = 172.20.1.20
NAS-Port = 1
NAS-Port-Id = "1"
State = 0x7eea823f76e09b79e9fd1595f5ebf47a
Called-Station-Id = "EC-E5-55-FF-FB-34:HIRSCHMANN_C"
NAS-Port-Type = Wireless-802.11
WLAN-RF-Band = 2
WLAN-Pairwise-Cipher = 1027076
WLAN-Group-Cipher = 1027076
WLAN-AKM-Suite = 1027073
Calling-Station-Id = "78-B8-D6-32-9A-32"
Connect-Info = "CONNECT 72 Mbps 802.11g/n"
NAS-Identifier = "AP_1"
Framed-MTU = 1500
EAP-Message = 0x020a002e190017030300230000000000000004425534a3fe3939c900e3f3f672628ae179e4a05c48c1964a31dba1
Message-Authenticator = 0xd87613c17b7b62950f6613cbca20e109
Event-Timestamp = "Apr 27 2021 09:37:46 CEST"
Timestamp = 1619509066
And if I perform a select from the radpostauth table I also see that the user has been successfully authenticated.
mysql> select * from radpostauth;
+----+----------+------+---------------+---------------------+
| id | username | pass | reply | authdate |
+----+----------+------+---------------+---------------------+
| 55 | demouser | | Access-Accept | 2021-04-27 09:04:19 |
| 56 | demouser | | Access-Accept | 2021-04-27 09:04:19 |
+----+----------+------+---------------+---------------------+
The problem is that in the radacct table no data is stored, when I perform a select it appears empty, I have created another table which is radaacct2 with the same structure and specified in the mysql configuration file but no data is inserted either.
From what I understand (correct me if I'm wrong) Radius stores accounting data in radacct table. I read this document
If no data is stored, the following query will not be executed correctly. /etc/freeradius/3.0/main/mysql/queries.conf :
#######################################################################
# Simultaneous Use Checking Queries
#######################################################################
# simul_count_query - query for the number of current connections
# - If this is not defined, no simultaneous use checking
# - will be performed by this module instance
simul_count_query = "\
SELECT COUNT(*) \
FROM ${acct_table1} \
WHERE username = '%{SQL-User-Name}' \
AND acctstoptime IS NULL"
In short, I need to control the simultaneous login of a specific user and the cause of not working correctly I think it is due to not inserting data into the mysql table. I am probably wrong because I am new to this, I ask please that in the case that it is so, someone provide me with information on how to fix this problem or specify me really what is the error in question.
I have been working on this for weeks and it is the last thing I need to configure but I can't figure it out.
If any more configuration files are needed please let me know and I will attach them.
Help please
PD: as extra information I have to add that I have followed the configuration steps on this page and it still does not work.
Thanks

Is a given sql statement vulnerable to a sql injection attack?

I'm building a sql statement like the one below in a rails app :-
bank_ids = params[:bank_ids] # comes from end user or simply, is a user input.
sql_string = "SELECT * FROM users WHERE bank_id IN (#{bank_ids});"
Is the sql statement above vulnerable to an injection attack, input 'bank_ids' is end user controlled.
Take as example a table designed to store a boolean value to tell if a user is admin or not (might not happend, but it's an example):
Table "public.users"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
------------+--------------------------------+-----------+----------+-----------------------------------+----------+--------------+-------------
id | bigint | | not null | nextval('users_id_seq'::regclass) | plain | |
name | character varying | | | | extended | |
admin | boolean | | | | plain | |
bank_id | integer | | | | plain | |
If you receive something like this:
'1) or id IN (select id from users where admin = true'
And that's interpolated afterward, then the select clause asking for the admin users will retrieve data that otherwise wouldn't appear. The query would be executed as it's built:
select * from users where bank_id IN (1) or id IN (select id from users where admin = true)
Is better for you to rely on the ORM you have at hand and leave it to do the sanitization and proper bindings for you (it's one of the reasons why those tools exist). Using ActiveRecord for example would bind the passed values for you, without having to do much:
User.where(bank_id: '1) or id IN (select id from users where admin = true')
# ... SELECT "users".* FROM "users" WHERE "users"."bank_id" = $1 [["bank_id", 1]]

find_in_set and find_in_set unexpected result

USE mysql;
DROP PROCEDURE IF EXISTS ShowUsers;
DELIMITER $
CREATE PROCEDURE `ShowUsers`(IN KnownUsers varchar(500), IN KnownHosts varchar(500))
BEGIN
SELECT
user,host
FROM
user
WHERE
NOT FIND_IN_SET(host, KnownHosts)
AND
NOT FIND_IN_SET(user, KnownUsers)
ORDER BY user, host ASC;
END $
DELIMITER ;
Example complete data to work with:
+-------------+-------------+
| user | host |
+-------------+-------------+
| knownuser1 | 192.168.1.5 |
| knownuser2 | 192.168.1.5 |
| unknownuser | 192.168.1.5 | # I want this result to show
| someuser1 | 192.168.1.6 |
| someuser2 | 192.168.1.6 |
| someuser3 | 192.168.1.6 |
| root | localhost |
+-------------+-------------+
I have marked the result I would want to show from running the procedure, basically the two IN parameters are known users, and known hosts those that should be have a user record on this database.
Calling the function like this
# users and hostnames(ips) to match for exclusion from results.
SET #Usernames = 'knownuser1,knownuser2';
SET #Hostnames = '192.168.1.5';
CALL ShowUsers(#Usernames, #Hostnames);
Expected Result:
+-------------+-------------+
| user | host |
+-------------+-------------+
| unknownuser | 192.168.1.5 | # I want this result to show
| someuser1 | 192.168.1.6 |
| someuser2 | 192.168.1.6 |
| someuser3 | 192.168.1.6 |
| root | localhost |
+-------------+-------------+
Actual Result:
+-------------+-------------+
| user | host |
+-------------+-------------+
| someuser1 | 192.168.1.6 |
| someuser2 | 192.168.1.6 |
| someuser3 | 192.168.1.6 |
| root | localhost |
+-------------+-------------+
Explanation (off this topic but I think I should clarify) The reason I want this procedure to work, I have a master server with multiple remote slaves, the slaves need to have access to the masters database which means they also have to have "root" access, they can create/reconfigure their own access credentials. The problem with this is if one of those servers were ever compromised it would leave open the chance to have a new user added with credentials to basically all of the database. Wide open and free to take.
I could lock the slaves out after initial configuration and manually open up the door, run an update and then lock it again which would be pretty laborious for the application and make the application virtually useless.
The idea I'm going with right now is to run this procedure via cron run script and check for unknown users/hosts and lock that slave server out of the database until I accept or reject the user from the main application.
The condition in the WHERE clause is:
NOT FIND_IN_SET(host, KnownHosts) AND NOT FIND_IN_SET(user, KnownUsers)
which is equivalent to:
NOT (FIND_IN_SET(host, KnownHosts) OR FIND_IN_SET(user, KnownUsers))
which means that you want to exclude the rows for which:
host is included in KnownHosts or user is included in KnownUsers.
So for your sample data, the row:
unknownuser | 192.168.1.5
will not be returned, because host = '192.168.1.5' and it is included in KnownHosts (= '192.168.1.5').
Maybe change the logical operator to OR, if this is the logic that you want to apply:
NOT FIND_IN_SET(host, KnownHosts) OR NOT FIND_IN_SET(user, KnownUsers)

MariaDB JOIN tables from different databases based on column value

How do I use a column value as the database name to JOIN two tables from those two different databases?
I have already successfully joined two tables between two databases with a statically defined (second) database name:
SELECT *
FROM db1.table_a AS ta
INNER JOIN db2.table_b AS tb on (ta.db_table_name = b.user_id)
However where db2.table_b is in that query I need to somehow have the db2 instead be a value from the first table in the first database; the table name will be statically defined. All of the kind-of-related threads were totally useless and wildly convoluted.
Details: there is one common database and all of the other databases represent the same application but for different accounts. In order for all of the users on all of the different accounts to be able to interact with each other (e.g. database_2.accounts.user.43 (DB->Table->Column->ID (43)) the common database (db1 above) must not only store the id of the user but also the name of the database that must be joined.
To help visualize things:
Database: common
Database: db2
SELECT id, database_name
FROM common.table_a AS ct
INNER JOIN [database_name].table_b AS dn ON (ct.user_id = [database_name].users.id)
Visually the data returned should look something like this:
+----------+------------+----------+
| database | account_id | username |
+----------+------------+----------+
| db1 | 1 | John |
+----------+------------+----------+
| db2 | 1 | Sally |
+----------+------------+----------+
| db3 | 43 | John |
+----------+------------+----------+
| db4 | 1 | Sally |
+----------+------------+----------+
Then the HTML output should look something like this:
Comment from John from db1.
Comment from Sally from db2.
Comment from John from db3.
Comment from Sally from db4.
I can worry about ensuring visually that John from db1 and John from db3 (and Sally from db2 and Sally from db4) all four of which are different people in real life are represented as so. It's the dynamic aspect of selecting them based on the value of the column's value that contains the database name to be used to JOIN is all that matters.
Do you have hundreds of databases? That would be a 'bad' design. To discuss further, please explain why you have so many.
If you don't have many databases, but you need to dynamically pick which db, again, poor design; let's discuss further.
If you must do one of those, the hide it in Stored Routines (as P.Salmon almost suggested; his code needs some polishing) or in an application library (PHP, Java, whatever).
Otherwise, wherever you can say table_a, you can replace that with db1.table_a. In fact, you can see MySQl doing that: EXPLAIN EXTENDED SELECT ...; SHOW WARNINGS; Example:
mysql> EXPLAIN EXTENDED SELECT province FROM canada; SHOW WARNINGS;
+----+-------------+--------+-------+---------------+----------+---------+------+------+----------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------+-------+---------------+----------+---------+------+------+----------+-------------+
| 1 | SIMPLE | canada | index | NULL | province | 105 | NULL | 5484 | 100.00 | Using index |
+----+-------------+--------+-------+---------------+----------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.01 sec)
+-------+------+---------------------------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+---------------------------------------------------------------------------------------+
| Note | 1003 | /* select#1 */ select `world`.`canada`.`province` AS `province` from `world`.`canada` |
+-------+------+---------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
In this example, the table canada was replaced by world.canada because world was the database.
All things being equal (and I bet they aren't) and assuming all schemas/dbs are on the same server you should be able to construct a simple dynamic sql statement.
so given
+----+------+
| id | name |
+----+------+
| 1 | aaa |
| 2 | bbb |
+----+------+
2 rows in set (0.00 sec)
Where name serves as proxy for db name we can first select all distinct names and create a sql statement unioning all the tables from all the dbs. Something like this.
SET #SQL =
(
SELECT GROUP_CONCAT(GCSTRING)
FROM
(
SELECT 'A' AS GC,CONCAT('SELECT ID,NAME FROM USERS U1 ',JSTRING,' ',DBNAME,' AS ',NAMEALIAS,' ON ',NAMEPREFIX,'.',USTRING) GCSTRING
FROM
(
SELECT DISTINCT 'JOIN ' AS JSTRING,NAME DBNAME ,
NAME AS NAMEALIAS, NAME AS NAMEPREFIX, 'TABLEB.USER_ID = UI.NAME UNION' USTRING
FROM USERS
) S
) T
GROUP BY GC
)
;
SET #SQL = REPLACE(#SQL,',',' ');
SET #SQL = SUBSTRING(#SQL,1,LENGTH(#SQL) - 5);
SET #SQL = CONCAT(#SQL,';');
SELECT #SQL
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| #SQL |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| SELECT ID NAME FROM USERS U1 JOIN aaa AS aaa ON aaa.TABLEB.USER_ID = UI.NAME UNION SELECT ID NAME FROM USERS U1 JOIN bbb AS bbb ON bbb.TABLEB.USER_ID = UI.NAME ; |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

MySQL to Redis - Import and Model

I'm thinking to use Redis to cache some user data snapshot(s) in order to speed up the access to that data (one of the reasons is because my MySQL table(s) suffer of lock contention) and I'm looking for the best way to import in one step a table like this(which may contain from a few record to millions of records):
mysql> select * from mytable where snapshot = 1133;
+------+--------------------------+----------------+-------------------+-----------+-----------+
| id | email | name | surname | operation | snapshot |
+------+--------------------------+----------------+-------------------+-----------+-----------+
| 2989 | example-2989#example.com | fake-name-2989 | fake-surname-2989 | 2 | 1133 |
| 2990 | example-2990#example.com | fake-name-2990 | fake-surname-2990 | 10 | 1133 |
| 2992 | example-2992#example.com | fake-name-2992 | fake-surname-2992 | 5 | 1133 |
| 2993 | example-2993#example.com | fake-name-2993 | fake-surname-2993 | 5 | 1133 |
| 2994 | example-2994#example.com | fake-name-2994 | fake-surname-2994 | 9 | 1133 |
| 2995 | example-2995#example.com | fake-name-2995 | fake-surname-2995 | 7 | 1133 |
| 2996 | example-2996#example.com | fake-name-2996 | fake-surname-2996 | 1 | 1133 |
+------+--------------------------+----------------+-------------------+-----------+-----------+
into the Redis key-value store.
I can have many "snapshots" to load into Redis, and the basic access pattern is (SQL like syntax)
select * from mytable where snapshot = ? and id = ?
these snapshots can also coming from others table, so the "global unique ID per snapshot" is the column snapshot, ex:
mysql> select * from my_other_table where snapshot = 1134;
+------+--------------------------+----------------+-------------------+-----------+-----------+
| id | email | name | surname | operation | snapshot |
+------+--------------------------+----------------+-------------------+-----------+-----------+
| 2989 | example-2989#example.com | fake-name-2989 | fake-surname-2989 | 1 | 1134 |
| 2990 | example-2990#example.com | fake-name-2990 | fake-surname-2990 | 8 | 1134 |
| 2552 | example-2552#example.com | fake-name-2552 | fake-surname-2552 | 5 | 1134 |
+------+--------------------------+----------------+-------------------+-----------+-----------+
The loaded snapshot into redis never change, they are available only for a week via TTL
There is a way to load in one step this kind of data(rows and columns) into redis combining redis-cli --pipe and HMSET?
What is the best model to use in redis in order to store/get this data (thinking at the access pattern)?
I have found the redis-cli --pipe Redis Mass Insertion (and also MySQL to Redis in One Step) but I can't figure out the best way to achieve my requirements (load from mysql in one step all rows/colums, best redis model for this) using HMSET
Thanks in advance
Cristian.
Model
To be able to query your data from Redis the same way as:
select * from mytable where snapshot = ?
select * from mytable where id = ?
You'll need the model below.
Note: select * from mytable where snapshot = ? and id = ? does not make a lot of sense here, since it's the same as select * from mytable where id = ?.
Key type and naming
[Key Type] [Key name pattern]
HASH d:{id}
ZSET d:ByInsertionDate
SET d:BySnapshot:{id}
Note: I used d: as a namespace but you may want to rename it with the name of your domain model.
Data insertion
Insert a new line from Mysql into Redis:
hmset d:2989 id 2989 email example-2989#example.com name fake-name-2989 ... snapshot 1134
zadd d:ByInsertionDate {current_timestamp} d:2989
sadd d:BySnapshot:1134 d:2989
Another example:
hmset d:2990 id 2990 email example-2990#example.com name fake-name-2990 ... snapshot 1134
zadd d:ByInsertionDate {current_timestamp} d:2990
sadd d:BySnapshot:1134 d:2990
Cron
Here is the algorithm that must be run each day or week depending on your requirements:
for key_name in redis(ZREVRANGEBYSCORE d:ByInsertionDate -inf {timestamp_one_week_ago})
// retrieve the snapshot id from d:{id}
val snapshot_id = redis(hget {key_name} snapshot)
// remove the hash (d:{id})
redis(del key_name)
// remove the hash entry from the set
redis(srem d:BySnapshot:{snapshot_id} {key_name})
// clean the zset from expired keys
redis(zremrangebyscore d:ByInsertionDate -inf {timestamp_one_week_ago})
Usage
select * from my_other_table where snapshot = 1134; will be either:
{snapshot_id} = 1134
for key_name in redis(smembers d:BySnapshot:{snapshot_id})
print(redis(hgetall {keyname}))
or write a lua script to do this directly on redis side. Finally:
select * from my_other_table where id = 2989; will be:
{id} = 2989
print(redis(hgetall d:{id}))
Import
This part is quite easy, just read the table and follow the above model. Depending on your requirements you may want to import all (or a part of) your data with an hourly/daily/weekly cron.