I referred this link to log access information into mysql database.
Below is my database structure for
apachelogs.web_access_log
+------------------+----------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------+----------------------+------+-----+---------+-------+
| id | char(19) | YES | | NULL | |
| agent | varchar(255) | YES | | NULL | |
| bytes_sent | int(10) unsigned | YES | | NULL | |
| child_pid | smallint(5) unsigned | YES | | NULL | |
| cookie | varchar(255) | YES | | NULL | |
| machine_id | varchar(25) | YES | | NULL | |
| request_file | varchar(255) | YES | | NULL | |
| referer | varchar(255) | YES | | NULL | |
| remote_host | varchar(50) | YES | | NULL | |
| remote_logname | varchar(50) | YES | | NULL | |
| remote_user | varchar(50) | YES | | NULL | |
| request_duration | smallint(5) unsigned | YES | | NULL | |
| request_line | varchar(255) | YES | | NULL | |
| request_method | varchar(10) | YES | | NULL | |
| request_protocol | varchar(10) | YES | | NULL | |
| request_time | char(28) | YES | | NULL | |
| request_uri | varchar(255) | YES | | NULL | |
| request_args | varchar(255) | YES | | NULL | |
| server_port | smallint(5) unsigned | YES | | NULL | |
| ssl_cipher | varchar(25) | YES | | NULL | |
| ssl_keysize | smallint(5) unsigned | YES | | NULL | |
| ssl_maxkeysize | smallint(5) unsigned | YES | | NULL | |
| status | smallint(5) unsigned | YES | | NULL | |
| time_stamp | int(10) unsigned | YES | | NULL | |
| virtual_host | varchar(255) | YES | | NULL | |
+------------------+----------------------+------+-----+---------+-------+
I see most of the column values are NULL as below when the request is received from browser:
+------+-------+------------+-----------+--------+------------+--------------+---------+-------------+----------------+-------------+------------------+--------------+----------------+------------------+--------------+-------------+--------------+-------------+------------+-------------+----------------+--------+------------+--------------+
| id | agent | bytes_sent | child_pid | cookie | machine_id | request_file | referer | remote_host | remote_logname | remote_user | request_duration | request_line | request_method | request_protocol | request_time | request_uri | request_args | server_port | ssl_cipher | ssl_keysize | ssl_maxkeysize | status | time_stamp | virtual_host |
+------+-------+------------+-----------+--------+------------+--------------+---------+-------------+----------------+-------------+------------------+--------------+----------------+------------------+--------------+-------------+--------------+-------------+------------+-------------+----------------+--------+------------+--------------+
| - | - | 0 | NULL | NULL | NULL | NULL | - | ::1 | NULL | - | 0 | NULL | NULL | HTTP/1.0 | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 408 | 1531980584 | 192.168.4.18 |
+------+-------+------------+-----------+--------+------------+--------------+---------+-------------+----------------+-------------+------------------+--------------+----------------+------------------+--------------+-------------+--------------+-------------+------------+-------------+----------------+--------+------------+--------------+
1 row in set (0.02 sec)
Most of the request parameters are missing. Let me know what I have to modify in order to log all the attributes with their corresponding values.
Thanks
Apache2 is unable to write the logs to MySQL database properly due to the mismatch of 'id' column width. The default width created in the web_access_log table is 19 which should be increased to 25. If 25 did not help try again with bigger width.
After doing this, the MySQL log automatically started updating without any further issues.
Related
Am using mysql 5.7.19-log server, i am working on application monitor tool where i need to monitor session details of a mysql server. In the process came to know that pid column can be null in sys.session table. i want to know what are the cases in which pid can be null?
when i exec desc session in mysql shell result as follows:
mysql>desc session;
+------------------------+------------------------------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+------------------------------------------+------+-----+---------+-------+
| thd_id | bigint(20) unsigned | NO | | NULL | |
| conn_id | bigint(20) unsigned | YES | | NULL | |
| user | varchar(128) | YES | | NULL | |
| db | varchar(64) | YES | | NULL | |
| command | varchar(16) | YES | | NULL | |
| state | varchar(64) | YES | | NULL | |
| time | bigint(20) | YES | | NULL | |
| current_statement | longtext | YES | | NULL | |
| statement_latency | text | YES | | NULL | |
| progress | decimal(26,2) | YES | | NULL | |
| lock_latency | text | YES | | NULL | |
| rows_examined | bigint(20) unsigned | YES | | NULL | |
| rows_sent | bigint(20) unsigned | YES | | NULL | |
| rows_affected | bigint(20) unsigned | YES | | NULL | |
| tmp_tables | bigint(20) unsigned | YES | | NULL | |
| tmp_disk_tables | bigint(20) unsigned | YES | | NULL | |
| full_scan | varchar(3) | NO | | | |
| last_statement | longtext | YES | | NULL | |
| last_statement_latency | text | YES | | NULL | |
| current_memory | text | YES | | NULL | |
| last_wait | varchar(128) | YES | | NULL | |
| last_wait_latency | text | YES | | NULL | |
| source | varchar(64) | YES | | NULL | |
| trx_latency | text | YES | | NULL | |
| trx_state | enum('ACTIVE','COMMITTED','ROLLED BACK') | YES | | NULL | |
| trx_autocommit | enum('YES','NO') | YES | | NULL | |
| pid | varchar(1024) | YES | | NULL | |
| program_name | varchar(1024) | YES | | NULL | |
+------------------------+------------------------------------------+------+-----+---------+-------+
28 rows in set (0.00 sec)```
In your screenshot, You need to check the column with the name Null. It says that whether the value of a field can be null or not. In your case, pid can be null because as per the description of a table, pid allows Null.
Possibilities of pid can be the null are followed.
Inserting pid as null value
INSERT INTO session (pid, program_name) values (null, 'test1'); // pid: null
Don't inserting pid
INSERT INTO session (program_name) values ('test1'); // pid: null
Updating pid as null value
UPDATE session SET pid = null WHERE `thd_id` = 1; // pid: null
I have a table called organization which has a column treeId. Tree id is used to manage wich organization is allowed to watch the profile of the other sub organizations. The values of the treeId in the table could be
1|2|44|23|56|12|
1|2|44|23|56|5|
1|2|42|21|52|
Now I want to show all organization name which comes after the searched organizationId also if I would look for 44 --> I want to show the organization name of 56, 12, 5 also all number after 23 without repetition. Is there a function in mysql to manage it?
organization table
+------------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+--------------+------+-----+---------+----------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| lastWrite | datetime | NO | | NULL | |
| status | varchar(1) | NO | | NULL | |
| name | varchar(100) | NO | | NULL | |
| city | varchar(100) | YES | | NULL | |
| email | varchar(100) | YES | | NULL | |
| homepage | varchar(100) | YES | | NULL | |
| mobile | varchar(30) | YES | | NULL | |
| nameAffix | varchar(100) | YES | | NULL | |
| notice | varchar(500) | YES | | NULL | |
| phone | varchar(30) | YES | | NULL | |
| poBox | varchar(20) | YES | | NULL | |
| street | varchar(100) | YES | | NULL | |
| zipCode | varchar(20) | YES | | NULL | |
| fax | varchar(30) | YES | | NULL | |
| providerId | bigint(20) | YES | MUL | NULL | |
| treeId | varchar(255) | YES | | NULL | |
| showCopyright | bit(1) | NO | | b'1' | |
| type | varchar(30) | YES | | NULL | |
| trackingUserInterface | varchar(20) | YES | | NULL | |
| country | varchar(20) | YES | | NULL | |
| iconStrategy | varchar(20) | YES | | NULL | |
| notificationInterval | bigint(20) | YES | | NULL | |
| refreshInterval | bigint(20) | YES | | NULL | |
| dateFormat | varchar(20) | YES | | NULL | |
| dateTimeFormat | varchar(20) | YES | | NULL | |
| locale | varchar(20) | YES | | NULL | |
| timeFormat | varchar(20) | YES | | NULL | |
| timeZone | varchar(20) | YES | | NULL | |
| defaultOrganization | bit(1) | YES | | NULL | |
| longDateFormat | varchar(20) | YES | | NULL | |
| showAllStatus | bit(1) | YES | | NULL | |
| coordinateFormat | varchar(20) | YES | | NULL | |
| params | text | YES | | NULL | |
| mapAndReportWidth | int(11) | YES | | NULL | |
| reverseGeocoderGroupId | bigint(20) | YES | MUL | NULL | |
| memo | text | YES | | NULL | |
| redirectAfterLogin | varchar(100) | YES | | NULL | |
+------------------------+--------------+------+-----+---------+----------------+
I want to get location from members IP. I can use the command on phpmyadmin, and it costs about 2 seconds. But it will be stuck when the command is executed on MySQL through terminal. How can I fix the problem.
SQL Command:
SELECT
ID,country
FROM
(SELECT ID,substring_index(members.server_ids,',',1) AS IP FROM members) AS A
JOIN
(SELECT country,start,end FROM ip) AS B
ON
INET_ATON(A.IP) BETWEEN INET_ATON(B.start) AND INET_ATON(B.end);
desc members:
+---------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+--------------+------+-----+---------+----------------+
| ID | int(11) | NO | PRI | NULL | auto_increment |
| USER_NAME | varchar(100) | NO | UNI | NULL | |
| PASSWORD | varchar(32) | YES | | NULL | |
| NAME | varchar(100) | YES | | NULL | |
| EMAIL | varchar(50) | YES | MUL | NULL | |
| SAFE_EMAIL | varchar(50) | YES | | NULL | |
| NICK_NAME | varchar(100) | YES | MUL | NULL | |
| GENDER | tinyint(1) | YES | | 0 | |
| BIRTHDAY | date | YES | | NULL | |
| CREATEDAY | datetime | YES | | NULL | |
| PHONE | varchar(20) | YES | | NULL | |
| MOBILE | varchar(15) | YES | | NULL | |
| ID_CARD | varchar(32) | YES | | NULL | |
| COUNTY | int(11) | YES | | NULL | |
| ADDRESS | varchar(255) | YES | | NULL | |
| LOGIN_TIME | datetime | YES | | NULL | |
| BUY_SN | varchar(50) | YES | | NULL | |
| BUY_KIND | varchar(50) | YES | | NULL | |
| PARTNER_SN | varchar(50) | YES | | NULL | |
| cookie_value | varchar(100) | YES | | NULL | |
| CONSIGNEE_INFO | int(11) | YES | MUL | NULL | |
| LOGIN_COUNT | int(11) | YES | | NULL | |
| PAY_COUNT | int(11) | YES | | NULL | |
| status | tinyint(4) | YES | | 1 | |
| experience | int(11) | YES | | 0 | |
| amount | int(11) | YES | | 0 | |
| score | int(11) | YES | | 0 | |
| is_facebook_account | tinyint(4) | YES | | 0 | |
| avatar | varchar(200) | YES | | NULL | |
| avatar_status | tinyint(1) | YES | | 0 | |
| member_authhash | varchar(50) | YES | | NULL | |
| job | varchar(50) | YES | | NULL | |
| server_ids | varchar(255) | YES | | NULL | |
+---------------------+--------------+------+-----+---------+----------------+
SELECT ID,server_ids FROM members LIMIT 10000,1;
+-------+--------------------------+
| ID | server_ids |
+-------+--------------------------+
| 20005 | 61.20.167.219, 127.0.0.1 |
+-------+--------------------------+
desc ip:
+---------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| country | char(2) | NO | | NULL | |
| type | char(4) | NO | | NULL | |
| start | varchar(15) | NO | MUL | NULL | |
| end | varchar(15) | NO | MUL | NULL | |
+---------+------------------+------+-----+---------+----------------+
SELECT * FROM ip LIMIT 100,1;
+-----+---------+------+-----------+---------------+
| id | country | type | start | end |
+-----+---------+------+-----------+---------------+
| 101 | IN | ipv4 | 1.187.0.0 | 1.187.255.255 |
+-----+---------+------+-----------+---------------+
phpMyAdmin automatically adds a LIMIT clause to the query. Because you do not have an ODER BY clause, MySQL sends the results as soon as it has found the number of records that you requested in the LIMIT clause.
Add a LIMIT clause to the query that you execute on the command line.
I have following table called transactions and I want to add foreign key using column account_id. But when I execute the query (see below) 0 rows affected. I am not getting what is wrong with the query.
I have two tables called transactions and accounts. Accounts table has an id as primary key and an account has_many transactions.
transactions table
+---------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| account_number | int(11) | YES | | NULL | |
| m_number | varchar(255) | YES | | NULL | |
| registration_number | varchar(255) | YES | | NULL | |
| page_number | varchar(255) | YES | | NULL | |
| entry_date | datetime | YES | | NULL | |
| narration | varchar(255) | YES | | NULL | |
| voucher_number | varchar(255) | YES | | NULL | |
| debit_credit | float | YES | | NULL | |
| profit | float | YES | | NULL | |
| account_code | int(11) | YES | | NULL | |
| balance | float | YES | | NULL | |
| branch | varchar(255) | YES | | NULL | |
| account_id | int(11) | YES | MUL | NULL | |
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
+---------------------+--------------+------+-----+---------+----------------+
16 rows in set (0,01 sec)
mysql> ALTER TABLE transactions ADD FOREIGN KEY (account_id) REFERENCES accounts(id);
Query OK, 0 rows affected (0,03 sec)
Records: 0 Duplicates: 0 Warnings: 0
accounts table
+-----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| account_number | int(11) | YES | | NULL | |
| account_code | int(11) | YES | | NULL | |
| branch | varchar(255) | YES | | NULL | |
| name | varchar(255) | YES | | NULL | |
| father_name | varchar(255) | YES | | NULL | |
| nic_number | varchar(255) | YES | | NULL | |
| mohalla | varchar(255) | YES | | NULL | |
| village | varchar(255) | YES | | NULL | |
| nominee | varchar(255) | YES | | NULL | |
| relationship | varchar(255) | YES | | NULL | |
| opening_balance | float | YES | | NULL | |
| opening_date | datetime | YES | | NULL | |
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
+-----------------+--------------+------+-----+---------+----------------+
After running the query shouldn't the column say F_K or something like PRI ?
Any help would be great. Thanks!
I need to normalise a table which contains cricket players:
+------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| matchID | int(11) | NO | PRI | 0 | |
| innings | int(11) | NO | PRI | 0 | |
| Bat1Name | varchar(30) | YES | | NULL | |
| Bat1Score | int(11) | YES | | NULL | |
| balls1 | int(11) | YES | | NULL | |
| sixes1 | int(11) | YES | | NULL | |
| fours1 | int(11) | YES | | NULL | |
| out1 | varchar(10) | YES | | NULL | |
| catcher1 | varchar(30) | YES | | NULL | |
| bowler1 | varchar(30) | YES | | NULL | |
| Bat2Name | varchar(30) | YES | | NULL | |
| Bat2Score | int(11) | YES | | NULL | |
| balls2 | int(11) | YES | | NULL | |
| sixes2 | int(11) | YES | | NULL | |
| fours2 | int(11) | YES | | NULL | |
| out2 | varchar(10) | YES | | NULL | |
| catcher2 | varchar(30) | YES | | NULL | |
| bowler2 | varchar(30) | YES | | NULL | |
| Bat3Name | varchar(30) | YES | | NULL | |
| Bat3Score | int(11) | YES | | NULL | |
| balls3 | int(11) | YES | | NULL | |
| sixes3 | int(11) | YES | | NULL | |
| fours3 | int(11) | YES | | NULL | |
| out3 | varchar(10) | YES | | NULL | |
| catcher3 | varchar(30) | YES | | NULL | |
| bowler3 | varchar(30) | YES | | NULL | |
| Bat4Name | varchar(30) | YES | | NULL | |
| Bat4Score | int(11) | YES | | NULL | |
| balls4 | int(11) | YES | | NULL | |
| sixes4 | int(11) | YES | | NULL | |
| fours4 | int(11) | YES | | NULL | |
| out4 | varchar(10) | YES | | NULL | |
| catcher4 | varchar(30) | YES | | NULL | |
| bowler4 | varchar(30) | YES | | NULL | |
| Bat5Name | varchar(30) | YES | | NULL | |
| Bat5Score | int(11) | YES | | NULL | |
| balls5 | int(11) | YES | | NULL | |
| sixes5 | int(11) | YES | | NULL | |
| fours5 | int(11) | YES | | NULL | |
| out5 | varchar(10) | YES | | NULL | |
| catcher5 | varchar(30) | YES | | NULL | |
| bowler5 | varchar(30) | YES | | NULL | |
| Bat6Name | varchar(30) | YES | | NULL | |
| Bat6Score | int(11) | YES | | NULL | |
| balls6 | int(11) | YES | | NULL | |
| sixes6 | int(11) | YES | | NULL | |
| fours6 | int(11) | YES | | NULL | |
| out6 | varchar(10) | YES | | NULL | |
| catcher6 | varchar(30) | YES | | NULL | |
| bowler6 | varchar(30) | YES | | NULL | |
| Bat7Name | varchar(30) | YES | | NULL | |
| Bat7Score | int(11) | YES | | NULL | |
| balls7 | int(11) | YES | | NULL | |
| sixes7 | int(11) | YES | | NULL | |
| fours7 | int(11) | YES | | NULL | |
| out7 | varchar(10) | YES | | NULL | |
| catcher7 | varchar(30) | YES | | NULL | |
| bowler7 | varchar(30) | YES | | NULL | |
| Bat8Name | varchar(30) | YES | | NULL | |
| Bat8Score | int(11) | YES | | NULL | |
| balls8 | int(11) | YES | | NULL | |
| sixes8 | int(11) | YES | | NULL | |
| fours8 | int(11) | YES | | NULL | |
| out8 | varchar(10) | YES | | NULL | |
| catcher8 | varchar(30) | YES | | NULL | |
| bowler8 | varchar(30) | YES | | NULL | |
| Bat9Name | varchar(30) | YES | | NULL | |
| Bat9Score | int(11) | YES | | NULL | |
| balls9 | int(11) | YES | | NULL | |
| sixes9 | int(11) | YES | | NULL | |
| fours9 | int(11) | YES | | NULL | |
| out9 | varchar(10) | YES | | NULL | |
| catcher9 | varchar(30) | YES | | NULL | |
| bowler9 | varchar(30) | YES | | NULL | |
| Bat10Name | varchar(30) | YES | | NULL | |
| Bat10Score | int(11) | YES | | NULL | |
| balls10 | int(11) | YES | | NULL | |
| sixes10 | int(11) | YES | | NULL | |
| fours10 | int(11) | YES | | NULL | |
| out10 | varchar(10) | YES | | NULL | |
| catcher10 | varchar(30) | YES | | NULL | |
| bowler10 | varchar(30) | YES | | NULL | |
| Bat11Name | varchar(30) | YES | | NULL | |
| Bat11Score | int(11) | YES | | NULL | |
| balls11 | int(11) | YES | | NULL | |
| sixes11 | int(11) | YES | | NULL | |
| fours11 | int(11) | YES | | NULL | |
| out11 | varchar(10) | YES | | NULL | |
| catcher11 | varchar(30) | YES | | NULL | |
| bowler11 | varchar(30) | YES | | NULL | |
| extras | int(11) | YES | | NULL | |
| wides | int(11) | YES | | NULL | |
| noBalls | int(11) | YES | | NULL | |
| byes | int(11) | YES | | NULL | |
| legByes | int(11) | YES | | NULL | |
| score | int(11) | YES | | NULL | |
| wickets | int(11) | YES | | NULL | |
| overs | float | YES | | NULL | |
| runrate | float | YES | | NULL | |
| team | varchar(100) | YES | | NULL | |
+------------+--------------+------+-----+---------+-------+
The rows I am interested in is all Bat?Name and Team fields eg. Bat1Name, Bat2Name....Bat11Name and the team they belong to.
I have created a player table:
mysql> describe players;
+----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+----------------+
| player_id | int(11) | NO | PRI | NULL | auto_increment |
| player_surname | varchar(30) | YES | | NULL | |
| team | varchar(100) | YES | MUL | NULL | |
+----------------+--------------+------+-----+---------+----------------+
3 rows in set (0.03 sec)
I successfully selected the player's names (without duplicates) into the player's table using the below UNION code:
INSERT INTO players (player_surname)
SELECT DISTINCT bat1Name from details
UNION
SELECT DISTINCT bat2Namen from details
UNION
SELECT DISTINCT bat3Name from details
(etc............................)
SELECT DISTINCT bat11Name from details;
This gave me 300 records which is correct.
But then realised I needed to include the teams so I added team:
INSERT INTO players (player_surname, team)
SELECT DISTINCT bat1Name,team from details
UNION
SELECT DISTINCT bat2Namen,team from details
UNION
SELECT DISTINCT bat3Name,team from details
(etc.............................)
SELECT DISTINCT bat11Name,team from details;
But this provides me with 310 records and says (for example) that Smith plays for UK, New Zealand and south africa but this is incorrect. It has also incorrectly pulled out a couple of others. I'm at a loss!
I assume my method of UNION is flawed so how can I get all batsmen out of the table with their corresponding team?
If there are two players with the same surnames that were playing in the different teams they will merge in the first query but not the second:
bat1 bat2 team
Smith Jones UK
Doe Smith Zealand
The first query will return Smith once, the second one twice.
It is most probable that your first query is wrong, not the second one. Try running this for your first query:
SELECT batname
FROM (
SELECT bat1Name AS batname, team
FROM details
UNION
SELECT bat2Name, team
FROM details
UNION …
) q
You don't need to do DISTINCT here: UNION will take care of this automatically.
As Quassnoi says it is probably the first query that is wrong as the second one will pull out the right data.
Also the player table should only store the player related data with a relation to another table storing the teams.