How To Left Join 2 Tables On 2 Different Databases? - mysql

I have first database (dbA) with table like this, named Username :
+------------------+--------------+
| Username | PhoneNumber |
+------------------+--------------+
| jamesbond007 | 555-0074 |
| batmanbegins | 555-0392 |
+------------------+--------------+
then, on the other side, I have dbB with table like this, named PrivateMessage :
+------------------+---------------------------------+
| Username | Message |
+------------------+---------------------------------+
| jamesbond007 | I need new bond-girl |
| batmanbegins | thanks for the paycheck, Nolan |
+------------------+---------------------------------+
now, how to combine this two tables from 2 different databases so the output will look like this :
+------------------+--------------+---------------------------------+
| Username | PhoneNumber | Message |
+------------------+--------------+---------------------------------+
| jamesbond007 | 555-0074 | I need new bond-girl |
| batmanbegins | 555-0392 | thanks for the paycheck, Nolan |
+------------------+--------------+---------------------------------+

You can simply join the table of different database. You need to specify the database name in your FROM clause. To make it shorter, add an ALIAS on it,
SELECT a.*, -- this will display all columns of dba.`UserName`
b.`Message`
FROM dba.`UserName` a -- or LEFT JOIN to show all rows whether it exists or not
INNER JOIN dbB.`PrivateMessage` b
ON a.`username` = b.`username`
but some how, there are possiblities where-in a username won't have messages. In this case use LEFT JOIN if you want still to show all the records of dba.Username.
Reading from your comments, the tables have different collation. The work around on this is to specify COLLATE on your joined statements,
SELECT a.*, -- this will display all columns of dba.`UserName`
b.`Message`
FROM dba.`UserName` COLLATE latin1_swedish_ci a
LEFT JOIN dbB.`PrivateMessage` COLLATE latin1_swedish_ci b
ON a.`username` = b.`username`
you can change latin1_swedish_ci to whatever you want.
For more info on COLLATION, see this full list of
Character Sets and Collations in MySQL
If you have enough privilege to ALTER the tables, simply use this syntax to manually convert and match their collations,
ALTER TABLE tbl_name CONVERT TO CHARACTER SET latin2 COLLATE 'latin2_general_ci';

Same as you would a normal table, except specifying the database:
SELECT dbA.Username, dbA.PhoneNumber, dbB.Message
FROM dbA.Username LEFT JOIN dbB.PrivateMessage
ON (dbA.UserName.Username = dbB.PrivateMessage.Username);
Things to look out for:
LEFT JOIN will return all users, also those with no messages (use INNER JOIN to retrieve only users with messages)
Users with multiple messages will appear multiple times (use aggregations and GROUP BY to only retrieve one message per user - you'll have to supply a criterion to choose the one message)
You need query privileges on both databases (otherwise some user with privileges on both has to copy, e.g. periodically in crontab, a table or a subset of a table from a database to the other)
Collations might not match. If this is the case, you have to change collation on one of the two tables using either COLLATE or converting the field of one DB to the charset of the other with CONVERT: CONVERT(db.table.field USING Latin1), which will prevent using indexes thus decreasing performances. You can modify one of the two tables, but verify that you're not disrupting whatever query or application is using the ALTER'ed table (in a pinch, convert the whole database to well-tempered UTF8).
JOINs on text fields aren't very efficient even if you have INDEX on that in both tables; it would be better to have the Message table holding a unique, numeric userid to refer to the message owner. I understand that two different databases with different logics might not be conducive to this solution, but you could apply one of the above "tricks" ("copy a table or subset thereof") and export, periodically, a converted and ID'ed table from a DB to the other. That one periodical query would be expensive, but all subsequent JOINs would greatly benefit.
Test run
This creates two tables with the same structure in two different databases, and joins them while in a third database.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.6.30 openSUSE package
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> CREATE DATABASE first_database;
Query OK, 1 row affected (0.01 sec)
mysql> CREATE DATABASE second_database;
Query OK, 1 row affected (0.00 sec)
mysql> USE first_database;
Database changed
mysql> CREATE TABLE mytable ( x integer, t varchar(32) );
Query OK, 0 rows affected (0.00 sec)
mysql> INSERT INTO mytable ( x, t ) VALUES ( 1, 'One in First Database' ), ( 2, 'Two in First Database' );
Query OK, 2 rows affected (0.00 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> USE second_database;
Database changed
mysql> CREATE TABLE mytable ( x integer, t varchar(32) );
Query OK, 0 rows affected (0.00 sec)
mysql> INSERT INTO mytable ( x, t ) VALUES ( 1, 'One in Second Database' ), ( 3, 'Three in Second Database' );
Query OK, 2 rows affected (0.00 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> USE test;
Database changed
mysql> SELECT * FROM first_database.mytable LEFT JOIN second_database.mytable USING ( x );
+------+-----------------------+------------------------+
| x | t | t |
+------+-----------------------+------------------------+
| 1 | One in First Database | One in Second Database |
| 2 | Two in First Database | NULL |
+------+-----------------------+------------------------+
2 rows in set (0.00 sec)
mysql>

The SQL for this is rather easy...
SELECT A.Username, A.PhoneNumber, B.Message
FROM dbA.Username as A
INNER JOIN dbB.PrivateMessage as B ON A.Username = B.Username
...assuming you can access both databases within your connection.
If you cannot access them, you have to work on a different approach (like copying one table to the other database before querying or something similar).

Try the below code
SELECT * FROM dbA.Username JOIN dbB.PrivateMessage USING(Username);

Related

Why do I get a different number of hits in a MySQL query if I use an index or not

I am new in MySQL and I do not understand, why if I use index on JSON column, result set is different as without index.
I have a simple table:
CREATE TABLE jsontest (
jsondata JSON
);
Table is filled up with 50000 JSONs and one filed in json is also:
"allowedNfTypes": ["aaa", "bbb", "ccc"]}
This field in some cases have 1 or 2 or 3 values in array (but there are about 10 string options - let's say from "aaa" to "iii").
In some cases this filed does not exist at all.
Then if I execute:
mysql> SELECT * FROM jsontest WHERE "AMF" MEMBER OF(jsondata->'$.allowedNfTypes')
I get:
10045 rows in set (0.21 sec)
Then I created an index:
CREATE INDEX allowedNfTypes_index ON jsontest((CAST(jsondata->'$.allowedNfTypes' AS CHAR(128) ARRAY)))
And the same query
SELECT * FROM jsontest WHERE "AMF" MEMBER OF(jsondata->'$.allowedNfTypes');
return much less hits:
1402 rows in set (0.03 sec)
Any idea why?
I have downloaded your data file.
CLI output copy:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 16
Server version: 8.0.23 MySQL Community Server - GPL
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> CREATE TABLE jsontest (
-> jsondata JSON
-> );
Query OK, 0 rows affected (0.07 sec)
mysql> LOAD DATA INFILE 'C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\test_data.json' INTO TABLE jsontest;
Query OK, 50000 rows affected (15.50 sec)
Records: 50000 Deleted: 0 Skipped: 0 Warnings: 0
mysql> SELECT COUNT(*) FROM jsontest WHERE "AMF" MEMBER OF(jsondata->'$.allowedNfTypes');
+----------+
| COUNT(*) |
+----------+
| 10045 |
+----------+
1 row in set (0.32 sec)
mysql> CREATE INDEX allowedNfTypes_index ON jsontest((CAST(jsondata->'$.allowedNfTypes' AS CHAR(128) ARRAY)));
Query OK, 0 rows affected (0.88 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> SELECT COUNT(*) FROM jsontest WHERE "AMF" MEMBER OF(jsondata->'$.allowedNfTypes');
+----------+
| COUNT(*) |
+----------+
| 10045 |
+----------+
1 row in set (0.28 sec)
Also original query (not SELECT COUNT(*) FROM .. but SELECT * FROM ..) was tested - the result is the same.
The issue was not reproduced.
After version updating till actual 8.0.28 the issue is reproduced. The amount of rows selected with the index is the same - 1402.
After adding autoincremented primary key ALTER TABLE jsontest ADD COLUMN id INT AUTO_INCREMENT PRIMARY KEY; the amount of rows selected with index alters to 1448.
Comparing the rows selected with and without index I have found that:
There is a lot of duplicates in jsondata->'$.allowedNfTypes'.
For each unique jsondata->'$.allowedNfTypes' value only the rows which have created unique id less than approximately 6800 are selected with the index by jsondata->'$.allowedNfTypes', all another rows with duplicated value are not selected.
Deleting the rows which have id value above some N decreases the amount of rows returned by the query without the index and does not alter the amount of rows returned by the query with index (tested N = 40000, 25000, 10000, 7500, 7000, 6800). When N is 6797 or less the amount of rows selected become equal.
I have tried to add generated column which extracts this property as JSON array with aLTER TABLE jsontest ADD COLUMN allowedNfTypes JSON AS (CAST(jsondata->'$.allowedNfTypes' AS JSON));, create multivalued index by this column and use this column insrtead of the whole JSON. The amount of rows selected with index increases till 1498. The value of N is 5097.
Then I add the index described by OP (the table contains 2 indices - by original column and by generated one) - and the query returns 10045! I drops the generated column and the index by it - 10045. I drop the index by original column and re-create it - again 1498.
There was a lot of additional experiments, but their results are less interesting.
Looks like a bug. Reproduceable bug. I think that you may repeat my expriment (and/or perform your own) and report this to MySQL bugtrack.

Really strange error on mysql query

I have this query, and I think it talks by itself:
mysql> select id,email from members where email LIKE "%abraham.sustaita#gmail.com%";
+--------+----------------------------+
| id | email |
+--------+----------------------------+
| 272118 | abraham.sustaita#gmail.com |
+--------+----------------------------+
1 row in set (0.69 sec)
mysql> select id,email from members where email = "abraham.sustaita#gmail.com";
Empty set (0.00 sec)
mysql> select id,email from members where id = 272118;
Empty set (0.00 sec)
The data exists, but it returns empty if I use other than LIKE...
When there is such a flagrant impossible sequence of queries, then it's time to think about a table (or index) corruption and to run the Mysql CHECK command.
In that case, running REPAIR TABLE members QUICK did the trick.
If the id is a varchar and the email is a varchar they might have surrounding spaces.

MySQL gender table field

If I want to create a gender field in my table, how do I make sure that my database doesn't accept any value apart from "M" or "F" ?
$sqlCommand = "CREATE TABLE members (
id int(11) NOT NULL auto_increment,
...
...
...
...
gender
)";
Thank you
No triggers, no enums or other deamonic activities.
You can use a FOREIGN KEY to a reference table with just 2 rows:
CREATE TABLE Gender_Ref
( gender CHAR(1) NOT NULL,
PRIMARY KEY (gender)
) ENGINE = InnoDB ;
INSERT INTO Gender_Ref (gender)
VALUES
('F'), ('M') ;
CREATE TABLE members
( id int(11) NOT NULL auto_increment,
...
...
gender CHAR(1) NOT NULL,
PRIMARY KEY (id),
FOREIGN KEY gender
REFERENCES Gender_Ref (gender)
) ENGINE = InnoDB ;
It's also good advice to "lock" the reference table so the applications code has only read access. (That's usually good for most reference tables, and if you have an Admin application, you can of course give it write access as well to the reference tables).
Like pointed out in the comments, you can use ENUM like so:
gender ENUM('F','M') NOT NULL
However, you have to be careful as this will still accept the empty string too (although you'll get a warning for that):
mysql> create table t (g enum('M','F') not null);
Query OK, 0 rows affected (0.12 sec)
mysql> insert into t values ('M');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t values ('');
Query OK, 1 row affected, 1 warning (0.00 sec)
mysql> show warnings;
+---------+------+----------------------------------------+
| Level | Code | Message |
+---------+------+----------------------------------------+
| Warning | 1265 | Data truncated for column 'g' at row 1 |
+---------+------+----------------------------------------+
1 row in set (0.00 sec)
mysql> select * from t;
+---+
| g |
+---+
| M |
| |
+---+
2 rows in set (0.00 sec)
To ensure this does not happen, you could consider setting the sql_mode (http://dev.mysql.com/doc/refman/5.5/en/server-sql-mode.html) to a more restrictive value:
mysql> set sql_mode = strict_all_tables;
Query OK, 0 rows affected (0.00 sec)
mysql> insert into t values ('');
ERROR 1265 (01000): Data truncated for column 'g' at row 1
However, you should investigate if this is a suitable option for you. Many existing applications (wordpress etc) don't like messing with the sql_mode so if your code is a plugin to those systems you want to avoid setting it.
You can choose to set the sql_mode server wide or session wide; The first option would be more robust, but requires configuring MySQL in a non default way, and is likely to affect other applications. Setting at the session level immediately after you open the connection should work just fine, but will clutter your application code. Pick your poison.
ypercube's suggestion to use a foreign key is also good, and is more portable to other RDBMSes than ENUM. However, you'll have to ensure your tables are both managed by the InnoDB engine. This is becoming more and more the standard so it's not a bad choice.
(if you're really paranoid, you should really ensure that the application only has read access to the gender reference table)
You could use enum type enum('M','F').
You can use a trigger to check if it is coreect or you can use an enum type enum('M','F')

MySQL query returns 0 rows when searching for value with dot (.) in string

If I try to search for a value in mysql database and the string value contains dot in it, query returns 0 rows. Example:
SELECT * FROM table WHERE `username`='marco.polo' --> 0 rows
SELECT * FROM table WHERE `username` LIKE '%.polo%' --> 0 rows
SELECT * FROM table WHERE `username` LIKE 'polo' --> Success
This appeared after moving server and database to another place. I know that dot is a set of extended regular expressions, but it should not apply to equal nor LIKE operator, simply because I don't use REGEXP in query.
I've tested the same query on my local database and it works fine.
Could there be a special setting in mysql that treats dot differently than it usually does?
user1084605, I tried to replicate the problem (using MySQL version 5.1.37), but got exactly the opposite results as you. See below:
mysql> create table test (username varchar(100));
Query OK, 0 rows affected (0.01 sec)
mysql> insert into test values ('marco.polo');
Query OK, 1 row affected (0.00 sec)
mysql> SELECT * FROM test WHERE `username`='marco.polo';
+------------+
| username |
+------------+
| marco.polo |
+------------+
1 row in set (0.00 sec)
mysql> SELECT * FROM test WHERE `username` LIKE '%.polo%';
+------------+
| username |
+------------+
| marco.polo |
+------------+
1 row in set (0.00 sec)
mysql> SELECT * FROM test WHERE `username` LIKE 'polo';
Empty set (0.00 sec)
According to the MySQL docs, the only special characters when using the LIKE operator are "%" (percent: matches 0, 1, or many characters) and "_" (underscore: matches one and only one character).
http://dev.mysql.com/doc/refman/5.0/en/string-comparison-functions.html
A "." (period) does have special meaning for MySQL's REGEXP operator, but it should still match a literal period in your column.
http://dev.mysql.com/doc/refman/5.0/en/regexp.html
Can you replicate the SQL statements I ran above and paste your results in reply?
As #cen already mentioned, character set can causes that problem.
I have had this sample:
`email` VARCHAR(45) CHARACTER SET 'armscii8' NOT NULL,
this is was in the .sql dump, which I receive.
So, when I was trying to fetch object with this email
I couldn't get it.
The below query takes care of the scenario when we have only DOT operator in the columns.
SELECT * FROM test WHERE `username` LIKE '%.%';

Attaching simple metadata to a MySQL database

Is there a way to attach a piece of metadata to a MySQL database? I'm trying to write code to automatically update the database schema whenever a code upgrade requires it. This requires the storage of a single integer value -- the schema version. I could of course create a whole table for it, but that seems like overkill for just a simple number.
You can use table comments to store the version:
ALTER TABLE table1 COMMENT = '1.4';
You'll have to regex to get the comment from this:
SHOW CREATE TABLE table1;
/COMMENT='(.*)'/
To answer the question as titled, that is for metadata for the entire database and not individual tables, there are a couple of choices, depending on the privileges that you have.
The most direct route is to create a stored function, which requires the CREATE ROUTINE privilege. e.g.
mysql> CREATE FUNCTION `mydb`.DB_VERSION() RETURNS VARCHAR(15)
RETURN '1.2.7.2861';
Query OK, 0 rows affected (0.03 sec)
mysql> SELECT `mydb`.DB_VERSION();
+--------------+
| DB_VERSION() |
+--------------+
| 1.2.7.2861 |
+--------------+
1 row in set (0.06 sec)
If your privileges limit you to only creating tables, you can create a simple table and put the metadata as default values. There’s no need to store any data in the table.
mysql> CREATE TABLE `mydb`.`db_metadata` (
`version` varchar(15) not null default '1.2.7.2861');
Query OK, 0 rows affected (0.00 sec)
mysql> SHOW COLUMNS FROM `mydb`.`db_metadata`;
+---------+-------------+------+-----+------------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------+-------------+------+-----+------------+-------+
| version | varchar(15) | NO | | 1.2.7.2861 | |
+---------+-------------+------+-----+------------+-------+
1 row in set (0.00 sec)