SQL Query Two Tables at once and Loop Result as JSON objects - mysql

I'm not exactly sure what I am looking for here, so apologies if this has already been covered here, I'm not sure what I need to search for!
I have a MySQL database with a table called "Locations" which looks a bit like this
id | name | other parameters
1 | shop1 | blah
2 | shop2 | blah
etc
and a table of customer queries
id | customer | department
1 | john | shop2
2 | Joe | shop2
3 | James | shop1
4 | Sue | shop2
etc
I want to query this and return a JSON object that looks like this
{"location":"shop1","queryCount":"1"},{"location":"shop2","queryCount":"3"}
The location table can be added to with time, and obviously the customer queries will be to, so both need dynamic queries.
I tried this by getting a list of locations by a simple SELECT name from locations query, turning it into an array and then looping through that as follows:
For i = UBound(listofLocations) To 0 Step -1
locations.id = listofLocations(i)
locations.queryCount= RESULT OF: "SELECT COUNT(id) as recordCount from queries WHERE department=listofLocations(i)"
objectArray.Add(locations)
Next
This works, but it is inefficient calling the database through the loop, how do I avoid this?
Thanks

The inefficient is because you are using nested query,
you should use LEFT JOIN, and you just need single query
Here is the SQL:-
select l.name, count(*) as recordCount
from Locations as l
left join customer as c
on l.name = c.department
group by l.id;
And your schema is not very optimized.
Your schema for customer should be
id, customer, location_id <-- using name is redundant,
<-- which should represent in ID (location ID)

First, I must recommend that your "department" field be changed. If you change the spelling of a name in the locations table, your relationships break.
Instead, use the id from the location table, and set up a foreign key reference.
But that's an aside. Using your current structure, this is what I'd do in SQL...
SELECT
location.name,
COUNT(queries.id) AS count_of_queries
FROM
locations
LEFT JOIN
queries
ON queries.department = locations.name
GROUP BY
location.name
Using a LEFT JOIN ensures that you get EVERY location, even if there is no query for it.
Using COUNT(queries.id) instead of COUNT(*) gives 0 if there are no associated records in the queries table.
You can then loop through the result-set of one query, rather than looping multiple queries, and build your JSON string.
It is possible to build the string in SQL, but that's generally considered bad practice. Much better to keep you SQL about data, and you php/whatever about processing and presenting it.

Try this:
SELECT T1.name, (SELECT COUNT(*) FROM queries AS T2 WHERE T2.department = T1.name) AS number FROM locations AS T1;
I agree that your scheme is not proper, you should reference department by Id, not by name

I think you're just looking for "GROUP BY"
SELECT
department_id as location,
COUNT(id) as querycount
FROM
(table of customer queries)
GROUP BY
department_id
As for the database structure... If possible, I would change the tables a bit:
Location:
id | name | other parameters
Customer:
id | name | whatever
table_of_customer_queries:
id | customer_id | location_id
1 | 2 | 2
2 | 4 | 2
3 | 2 | 1
4 | 1 | 2
GROUP BY will only give you results for those departments that have queries. If you want all departments, the LEFT JOIN option mentioned earlier is the way to go.

First off, if your Locations table really stores Department information, rename it as such. Then, change Customer.department to be a fk reference to the Locations.id column (and renamed to department_id), not the name (so much less to get wrong). This may or may not give you a performance boost; however, keep in mind that the data in the database isn't really meant to be human readable - it's meant to be program readable.
In either case, the query can be written as so:
SELECT a.name, (SELECT COUNT(b.id)
FROM Customer as b
WHERE b.department_id = a.id) as count
FROM Location as a

If you don't change your schema, your answer is to do a group by clause that counts the number of queries to each location.
Creating tables like yours:
create table #locations (id integer, name varchar(20), other text)
create table #queries (id integer, customer varchar(20), department varchar(20))
insert into #locations (id, name, other) values (1, 'shop1', 'blah')
insert into #locations (id, name, other) values (2, 'shop2', 'blah')
insert into #queries (id, customer, department) values (1, 'john', 'shop2')
insert into #queries (id, customer, department) values (2, 'Joe', 'shop2')
insert into #queries (id, customer, department) values (3, 'James', 'shop1')
insert into #queries (id, customer, department) values (4, 'Sue', 'shop2')
Querying your data:
select
l.name as location,
count(q.id) as queryCount
from #locations as l
left join #queries as q on q.department = l.name
group by l.name
order by l.name
Results:
location queryCount
-------------------- -----------
shop1 1
shop2 3

Related

MySQL Query using a Foreign Key to return Names in Ordered Format

I have a query related to fetching records from the combination of 2 tables in a way that the returned result will be fetched using the ORDER by clause with the help of foreign key.
I have two tables named users and orders.
Table: users
id name
1 John
2 Doe
Table: orders
id user_id for
1 2 cake
2 1 shake
2 2 milk
In table:orders, user_id is foreign key representing id in table:users.
Question:
I want to extract the records from table:orders but the ORDER should be based on name of users.
Desired Results:
user_id for
2 cake
2 milk
1 shake
Note: Here user_id with 2 is showing before user id with 1. This is because the name Doe should be shown before the name John because of order by.
What I have done right now:
I have no idea about MySQL joins. By searching this thing on the internet i did not find a way how i will achieve this thing. I have written a query but it will not fetch such record but have no idea what should i do to make it work exactly like what i want to.
SELECT * FROM orders ORDER BY user_id
It will fetch the records according to the order of user_id but not with name.
you are right join both tables is the simplest way to achieve that and you can show the names also, as you have them joined anyway
CREATE TABLE orders (
`id` INTEGER,
`user_id` INTEGER,
`for` VARCHAR(5)
);
INSERT INTO orders
(`id`, `user_id`, `for`)
VALUES
('1', '2', 'cake'),
('2', '1', 'shake'),
('2', '2', 'milk');
CREATE TABLE users (
`id` INTEGER,
`name` VARCHAR(4)
);
INSERT INTO users
(`id`, `name`)
VALUES
('1', 'John'),
('2', 'Doe');
SELECT o.`user_id`, o.`for` FROM orders o INNER JOIN users u ON u.id = o.user_id ORDER BY u.name
user_id | for
------: | :----
2 | cake
2 | milk
1 | shake
db<>fiddle here
You can get your desired results by join orders table and users table by simply using below query.
SELECT user_id, for FROM orders, users where user_id = id ORDER BY name;
Using where condition, we match corresponding rows where user_id in orders table equals id in users table. By using ORDER BY for name column in users table, rows will be sorted in ascending order. Here user_id and for columns in orders table will be show as final result.
Here I haven't use users.id or orders.user_id because they are in different formats. If you use same format for columns, you need to use above syntax.

Use another query with the same result

I have query:
SELECT Name, Surname, Telephone, PersonID FROM Client WHERE Telephone =
(SELECT MAX(Telephone) FROM Client) OR PersonID = (SELECT MAX(PersonID)
FROM Client);
With result:
| Name | Surname | Telephone | PersonID |
| Tyler | Henry | 998778781 | 38568215856 |
| Brooke | Thornton | 617196573 | 99412132661 |
What other query will give me the same result? I have to compare two queries in terms of optimization.
You can use a UNION optimization:
SELECT Name, Surname, Telephone, PersonID
FROM Client
WHERE Telephone = (SELECT MAX(Telephone) FROM Client)
UNION
SELECT Name, Surname, Telephone, PersonID
FROM Client
WHERE PersonID = (SELECT MAX(PersonID) FROM Client)
If a subset of selected columns is UNIQUE, the query will return the same result.
Given indexes on (Telephone) and (PersonID) the query should also be much faster on a big table. The reason is that MySQL is not able to use more than one index per table and (sub)query. Splitting the query in two subqueries will allow the engine to use both indexes.
You can get the max Telephone and max PersonID in a single query and then join it to the table:
SELECT c.Name, c.Surname, c.Telephone, c.PersonID
FROM Client c INNER JOIN (
SELECT MAX(Telephone) Telephone, MAX(PersonID) PesronID FROM Client
)m ON m.Telephone = c.Telephone OR m.PersonID = c.PersonID
or with NOT EXISTS:
SELECT Name, Surname, Telephone, PersonID FROM Client c
WHERE
NOT EXISTS (
SELECT 1 FROM Client
WHERE Telephone > c.Telephone
)
OR
NOT EXISTS (
SELECT 1 FROM Client
WHERE PersonID > c.PersonID
)
It depends on what indexes you have. Without INDEX(Telephone) and INDEX(PersonID) most of the answers will do table scan(s).
Here's another contender:
( SELECT Name, Surname, Telephone, PersonID
FROM Client
ORDER BY Telephone DESC LIMIT 1 )
UNION DISTINCT
( SELECT Name, Surname, Telephone, PersonID
FROM Client
ORDER BY PersonID DESC LIMIT 1 )
This may be faster because it does not have subqueries. However, it may give different results if two have the same Telephone or PersonID.
I suggest you run all the formulations against your data to see which is best.
If you want to discuss things further, please provide SHOW CREATE TABLE client and EXPLAIN SELECT ... -- both will give clues of what will/won't run faster.

Mysql. How to select records with mutiple parameters (logical AND) [duplicate]

I have a very narrow table: user_id, ancestry.
The user_id column is self explanatory.
The ancestry column contains the country from where the user's ancestors hail.
A user can have multiple rows on the table, as a user can have ancestors from multiple countries.
My question is this: how do I select users whose ancestors hail from multiple, specified countries?
For instance, show me all users who have ancestors from England, France and Germany, and return 1 row per user that met that criteria.
What is that SQL?
user_id ancestry
--------- ----------
1 England
1 Ireland
2 France
3 Germany
3 Poland
4 England
4 France
4 Germany
5 France
5 Germany
In the case of the data above, I would expect the result to be "4" as user_id 4 has ancestors from England, France and Germany.
To clarify: Yes, the user_id / ancestry columns make a unique pair, so a country would not be repeated for a given user. I am looking for users who hail from all 3 countries - England, France, AND Germany (and the countries are arbitrary).
I am not looking for answers specific to a certain RDBMS. I'm looking to answer this problem "in general."
I'm content with regenerating the where clause for each query provided generating the where clause can be done programmatically (e.g. that I can build a function to build the WHERE / FROM - WHERE clause).
Try this:
Select user_id
from yourtable
where ancestry in ('England', 'France', 'Germany')
group by user_id
having count(user_id) = 3
The last line means the user's ancestry has all 3 countries.
SELECT DISTINCT (user_id)
FROM [user]
WHERE user.user_id In (select user_id from user where ancestry = 'England')
And user.user_id In (select user_id from user where ancestry = 'France')
And user.user_id In (select user_id from user where ancestry = 'Germany');`
Users who have one of the 3 countries
SELECT DISTINCT user_id
FROM table
WHERE ancestry IN('England','France','Germany')
Users who have all 3 countries
SELECT DISTINCT A.userID
FROM table A
INNER JOIN table B on A.user_id = B.user_id
INNER JOIN table C on A.user_id = C.user_id
WHERE A.ancestry = 'England'
AND B.ancestry = 'Germany'
AND C.ancestry = 'France'
This question is some years old but i came via a duplicate to it. I want to suggest a more general solution too. If you know you always have a fixed number of ancestors you can use some self joins as already suggested in the answers. If you want a generic approach go on reading.
What you need here is called Quotient in relational Algebra. The Quotient is more or less the reversal of the Cartesian Product (or Cross Join in SQL).
Let's say your ancestor set A is (i use a table notation here, i think this is better for understanding)
ancestry
-----------
'England'
'France'
'Germany'
and your user set U is
user_id
--------
1
2
3
The cartesian product C=AxU is then:
user_id | ancestry
---------+-----------
1 | 'England'
1 | 'France'
1 | 'Germany'
2 | 'England'
2 | 'France'
2 | 'Germany'
3 | 'England'
3 | 'France'
3 | 'Germany'
If you calculate the set quotient U=C/A then you get
user_id
--------
1
2
3
If you redo the cartesian product UXA you will get C again. But note that for a set T, (T/A)xA will not necessarily reproduce T. For example, if T is
user_id | ancestry
---------+-----------
1 | 'England'
1 | 'France'
1 | 'Germany'
2 | 'England'
2 | 'France'
then (T/A) is
user_id
--------
1
(T/A)xA will then be
user_id | ancestry
---------+------------
1 | 'England'
1 | 'France'
1 | 'Germany'
Note that the records for user_id=2 have been eliminated by the Quotient and Cartesian Product operations.
Your question is: Which user_id has ancestors from all countries in your ancestor set? In other words you want U=T/A where T is your original set (or your table).
To implement the quotient in SQL you have to do 4 steps:
Create the Cartesian Product of your ancestry set and the set of
all user_ids.
Find all records in the Cartesian Product which have no partner in the original set (Left Join)
Extract the user_ids from the resultset of 2)
Return all user_ids from the original set which are not included in the result set of 3)
So let's do it step by step. I will use TSQL syntax (Microsoft SQL server) but it should easily be adaptable to other DBMS. As a name for the table (user_id, ancestry) i choose ancestor
CREATE TABLE ancestry_set (ancestry nvarchar(25))
INSERT INTO ancestry_set (ancestry) VALUES ('England')
INSERT INTO ancestry_set (ancestry) VALUES ('France')
INSERT INTO ancestry_set (ancestry) VALUES ('Germany')
CREATE TABLE ancestor ([user_id] int, ancestry nvarchar(25))
INSERT INTO ancestor ([user_id],ancestry) VALUES (1,'England')
INSERT INTO ancestor ([user_id],ancestry) VALUES(1,'Ireland')
INSERT INTO ancestor ([user_id],ancestry) VALUES(2,'France')
INSERT INTO ancestor ([user_id],ancestry) VALUES(3,'Germany')
INSERT INTO ancestor ([user_id],ancestry) VALUES(3,'Poland')
INSERT INTO ancestor ([user_id],ancestry) VALUES(4,'England')
INSERT INTO ancestor ([user_id],ancestry) VALUES(4,'France')
INSERT INTO ancestor ([user_id],ancestry) VALUES(4,'Germany')
INSERT INTO ancestor ([user_id],ancestry) VALUES(5,'France')
INSERT INTO ancestor ([user_id],ancestry) VALUES(5,'Germany')
1) Create the Cartesian Product of your ancestry set and the set of all user_ids.
SELECT a.[user_id],s.ancestry
FROM ancestor a, ancestry_set s
GROUP BY a.[user_id],s.ancestry
2) Find all records in the Cartesian Product which have no partner in the original set (Left Join) and
3) Extract the user_ids from the resultset of 2)
SELECT DISTINCT cp.[user_id]
FROM (SELECT a.[user_id],s.ancestry
FROM ancestor a, ancestry_set s
GROUP BY a.[user_id],s.ancestry) cp
LEFT JOIN ancestor a ON cp.[user_id]=a.[user_id] AND cp.ancestry=a.ancestry
WHERE a.[user_id] is null
4) Return all user_ids from the original set which are not included in the result set of 3)
SELECT DISTINCT [user_id]
FROM ancestor
WHERE [user_id] NOT IN (
SELECT DISTINCT cp.[user_id]
FROM (SELECT a.[user_id],s.ancestry
FROM ancestor a, ancestry_set s
GROUP BY a.[user_id],s.ancestry) cp
LEFT JOIN ancestor a ON cp.[user_id]=a.[user_id] AND cp.ancestry=a.ancestry
WHERE a.[user_id] is null
)
First way: JOIN:
get people with multiple countries:
SELECT u1.user_id
FROM users u1
JOIN users u2
on u1.user_id = u2.user_id
AND u1.ancestry <> u2.ancestry
Get people from 2 specific countries:
SELECT u1.user_id
FROM users u1
JOIN users u2
on u1.user_id = u2.user_id
WHERE u1.ancestry = 'Germany'
AND u2.ancestry = 'France'
For 3 countries... join three times. To only get the result(s) once, distinct.
Second way: GROUP BY
This will get users which have 3 lines (having...count) and then you specify which lines are permitted. Note that if you don't have a UNIQUE KEY on (user_id, ancestry), a user with 'id, england' that appears 3 times will also match... so it depends on your table structure and/or data.
SELECT user_id
FROM users u1
WHERE ancestry = 'Germany'
OR ancestry = 'France'
OR ancestry = 'England'
GROUP BY user_id
HAVING count(DISTINCT ancestry) = 3
one of the approach if you want to get all user_id that satisfies all conditions is:
SELECT DISTINCT user_id FROM table WHERE ancestry IN ('England', '...', '...') GROUP BY user_id HAVING count(*) = <number of conditions that has to be satisfied>
etc. If you need to take all user_ids that satisfies at least one condition, then you can do
SELECT DISTINCT user_id from table where ancestry IN ('England', 'France', ... , '...')
I am not aware if there is something similar to IN but that joins conditions with AND instead of OR
brute force (and only tested on an Oracle system, but I think this is pretty standard):
select distinct usr_id from users where user_id in (
select user_id from (
Select user_id, Count(User_Id) As Cc
From users
GROUP BY user_id
) Where Cc =3
)
and ancestry in ('England', 'France', 'Germany')
;
edit: I like #HuckIt's answer even better.
like the answer above but I have a duplicate record so I have to create a subquery with distinct
Select user_id
(
select distinct userid
from yourtable
where user_id = #userid
) t1
where
ancestry in ('England', 'France', 'Germany')
group by user_id
having count(user_id) = 3
this is what I used because I have multiple record(download logs) and this checks that all the required files have been downloaded
I was having a similar issue like yours, except that I wanted a specific subset of 'ancestry'. Hong Ning's query was a good start, except it will return combined records containing duplicates and/or extra ancestries (e.g. it would also return someone with ancestries ('England', 'France', 'Germany', 'Netherlands') and ('England', 'France', 'England'). Supposing you'd want just the three and only the three, you'd need the following query:
SELECT Src.user_id
FROM yourtable Src
WHERE ancestry in ('England', 'France', 'Germany')
AND EXISTS (
SELECT user_id
FROM dbo.yourtable
WHERE user_id = Src.user_id
GROUP BY user_id
HAVING COUNT(DISTINCT ancestry) = 3
)
GROUP BY user_id
HAVING COUNT(DISTINCT ancestry) = 3

How to get counts using the `IN` operator

I am trying to use the IN operator to get the count of certain fields in the table.
This is my query:
SELECT order_id, COUNT(*)
FROM remake_error_type
WHERE order_id IN (1, 2, 100)
GROUP BY order_id;
My current output:
| order_id | COUNT(*) |
+----------+----------+
| 1 | 8 |
| 2 | 8 |
My expected output:
| order_id | COUNT(*) |
+----------+----------+
| 1 | 8 |
| 2 | 8 |
| 100 | 0 |
You can write your query this way:
SELECT t.id, COUNT(remake_error_type.order_id)
FROM
(SELECT 1 AS id UNION ALL SELECT 2 UNION ALL SELECT 100) as t
LEFT JOIN remake_error_type
ON t.id = remake_error_type.order_id
GROUP BY
t.id
a LEFT JOIN will return all rows from the subquery on the left, and the COUNT(remake_error_type.order_id) will count all values where the join succeeds.
You can create a temporary table, insert as many order_ids as required, and perform the left join to remake_error_type. At a small number of orders the other answers are sufficient, but if you were doing this for a lot of orders, UNION ALL and sub-queries are inefficient, both to type it up and to execute on the server.
Additionally, this is a very dynamic approach, because you can control easily the values in your temp table by modifying the insert statement.
However, this will only work if the database user has sufficient privileges: at least select, create temporary and drop table.
DROP TABLE IF EXISTS myTempOrders;
CREATE TEMPORARY TABLE myTempOrders (order_id INTEGER, PRIMARY KEY(order_id));
INSERT INTO myTempOrders (order_id) VALUES (1), (2), (100);
SELECT temp.order_id, count(*)
FROM myTempOrders temp
LEFT JOIN remake_error_type ON temp.order_id = remake_error_type.order_id
GROUP BY 1
If the order_id values exist in some table, then it is possible to extract the desired result without creating a temporary table and inserting values into it.
To qualify, the table must
have an auto increment primary key with # rows greater than the maximum sought order_id value
have a starting increment value less than the minimum sought order_id value
have no missing values in the primary key (i.e. no records have been deleted)
if a qualified table exists, then you can run the following query, where you have to replace surrogate with the qualified table name and surrogate_id with the auto-incrementing primary key of the qualified table name
SELECT surrogate.surrogate_id, count(*)
FROM my_qualified_table surrogate
LEFT JOIN remake_error_type ON surrogate.surrogate_id = remake_error_type.order_id
WHERE surrogate.surrogate_id IN (1, 2, 100)
GROUP BY 1
You could use a union for this. No, this does not use the IN operator, but it is an alternative that will give you your expected results. One option is to hardcode the order_id and use conditional aggregation to get the SUM() of rows with that id:
SELECT 1 AS order_id, SUM(order_id = 1) AS numOrders FROM myTable
UNION ALL
SELECT 2 AS order_id, SUM(order_id = 2) AS numOrders FROM myTable
UNION ALL
SELECT 100 AS order_id, SUM(order_id = 100) AS numOrders FROM myTable;
Here is an SQL Fiddle example.

Find which values are not in table

Simple question, but I'm drawing a blank. Any help is appreciated.
I have a table of ids:
-------
| ids |
-------
| 1 |
| 5 |
| 7 |
-------
Except the actual table is thousands of entries long.
I have a list (x), not a table, of other ids, say 2, 6, 7. I need to see which ids from x are not in the ids table.
I need to get back (2,6).
I tried something like this:
SELECT id FROM ids WHERE id IN (2,6,7) GROUP BY id HAVING COUNT(*) = 0;
However, COUNT(*) returns count of retrieved rows only, it doesn't return 0.
Any suggestions?
Create a temporary table, insert the IDs that you need into it, and run a join, like this:
CREATE TEMPORARY TABLE temp_wanted (id BIGINT);
INSERT INTO temp_wanted(id) VALUES (2),(6),(7);
SELECT id
FROM temp_wanted t
LEFT OUTER JOIN ids i ON i.id=t.id
WHERE i.id IS NULL
Try something with "NOT IN" clause:
select * from
(SELECT 2 as id
UNION ALL
SELECT 6 as id
UNION ALL
SELECT 7 as id) mytable
WHERE ID not in (SELECT id FROM ids)
See fiddle here