Related
I have seven tables (from XT1 to XT7) and each have the same structure as below:
Date
m1
m2
m3
2021-06-01
4
2
6
2021-06-02
3
2
5
2021-06-03
12
2
14
.....
..
..
..
I only need m3 value from each table. What I'm trying to do, is to show all m3 values in a single table and sum them up like below:
Date
XT1.m3
XT2.m3
XT3.m3
XT4.m3
XT5.m3
XT6.m3
XT7.m3
Subtotal XT1~7.m3
2021-06-01
6
7
8
6
7
8
8
50
2021-06-02
6
7
8
6
7
8
8
50
2021-06-03
6
7
8
6
7
8
8
50
Total
18
21
24
18
21
24
24
150
What I have tried before:
select (TX1.m3+TX2.m3+TX3.m3...) AS subtotal, date_format(date, '%Y-%m') as date,TX1.m3,TX2.m3,TX3.m3
from TX1 AS c1 left join TX2 AS c2 on TX1.date=TX2.date
left join TX3 AS c3 on TX2.date=TX3.date
Which was modified from a previous working script. But the old structure only had 3 tables and now I have seven.
UPDATE:Tried the following
SELECT TX1.date,TX1.m3,TX2.m3,TX2.m3...TX7.m3
FROM TX1
inner JOIN TX2 ON TX1.date = TX2.date
inner JOIN TX3 ON TX2.date = TX3.date
......
inner JOIN TX7 ON TX6.date = TX7.date
Above code will return an un-grouped matrix. Tried to group with 'date' and returned error 1055. Override with
SET sql_mode=(SELECT REPLACE(##sql_mode,'ONLY_FULL_GROUP_BY',''));
Now works, but not sure if this is the correct way to do. Any ideas?
If you want to use the same query structure with addition of newly created tables, you can try using prepared statement. For example:
Preparing field values:
/*setting variables as NULL*/
SET #sql = NULL;
SET #tbl = NULL;
SET #val1 = NULL;
SET #val2 = NULL;
SET #reftbl = 'TX1'; /*this is your reference (most left) table in the LEFT JOIN*/
/*setting each variables with values*/
SELECT GROUP_CONCAT(CONCAT(table_name,'.',column_name,' AS ',table_name,column_name) SEPARATOR ', ') INTO #val1
FROM information_schema.columns
WHERE table_name LIKE 'TX%'
#AND table_schema=your_database_name
AND column_name='m3';
SELECT CONCAT(GROUP_CONCAT(CONCAT(table_name,'.',column_name) ORDER BY table_name SEPARATOR '+' ),' as subtotal') INTO #val2
FROM information_schema.columns
WHERE table_name LIKE 'TX%'
#AND table_schema=your_database_name
AND column_name='m3'
Here we are using information_schema.columns tables to generate field of all the m3 columns coming from all tables that correspond to the condition (table_name and table_schema). This will return you something like:
#val1: TX1.m3 AS TX1m3, TX2.m3 AS TX2m3, TX3.m3 AS TX3m3, TX4.m3 AS TX4m3, TX5.m3 AS TX5m3, TX6.m3 AS TX6m3, TX7.m3 AS TX7m3
#val2:TX1.m3+TX2.m3+TX3.m3+TX4.m3+TX5.m3+TX6.m3+TX7.m3 as subtotal
Next is to prepare the tables required in the query and add LEFT JOIN.
SELECT CONCAT(#reftbl,
GROUP_CONCAT(
CASE WHEN tbn IS NOT NULL
THEN CONCAT(' LEFT JOIN ', table_name,' ON ',tbn,'.date =',table_name,'.date')
ELSE table_name END SEPARATOR ' ')) INTO #tbl
FROM
(SELECT #reftbl tbn,
table_name
FROM information_schema.tables
WHERE table_name LIKE 'TX%'
#AND table_schema=your_database_name
AND table_name <> #reftbl) v;
The operation I'm doing here is to generate the LEFT JOIN parts by using CONCAT and GROUP_CONCAT.
Then we set #sql variable with a query generated based on all the variables we previously set; constructed using CONCAT.
/*constructing query and set into #sql*/
SET #sql = CONCAT('SELECT DATE_FORMAT(TX1.DATE, "%Y-%m") AS DATE,',#val1,',',#val2,'
FROM ',#tbl,';');
This will end up with a query like:
SELECT DATE_FORMAT(TX1.DATE, "%Y-%m") AS DATE,TX1.m3 AS TX1m3, TX2.m3 AS TX2m3,
TX3.m3 AS TX3m3, TX4.m3 AS TX4m3, TX5.m3 AS TX5m3, TX6.m3 AS TX6m3, TX7.m3 AS TX7m3,
TX1.m3+TX2.m3+TX3.m3+TX4.m3+TX5.m3+TX6.m3+TX7.m3 as subtotal
FROM TX1 LEFT JOIN TX2 ON TX1.date =TX2.date
LEFT JOIN TX3 ON TX1.date =TX3.date
LEFT JOIN TX4 ON TX1.date =TX4.date
LEFT JOIN TX5 ON TX1.date =TX5.date
LEFT JOIN TX6 ON TX1.date =TX6.date
LEFT JOIN TX7 ON TX1.date =TX7.date;
Lastly, prepare, execute then deallocate the #sql statement and you'll get the desired result you're looking for:
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
This is a way of generating 'dynamic query' which means if/when you add/remove tables, the query will be generated with the additional or without the removed tables. For example, if there's a new table TX8, when you run the queries above, it will include TX8 in the prepared statement as long as it match the condition.
Here's a demo fiddle that includes a situation when there's a new table created.
In another database, this problem would be solved with multiple FULL joins, but since MySql does not support them and simulating them for 7 tables is not an option, I would choose UNION ALL.
From each table, return 8 columns: 1 for the date, 1 for m3 and 6 null columns.
Use UNION ALL to combine them and finally aggregate:
SELECT date,
MAX(m3_1) m3_1, MAX(m3_2) m3_2, MAX(m3_3) m3_3, MAX(m3_4) m3_4, MAX(m3_5) m3_5, MAX(m3_6) m3_6, MAX(m3_7) m3_7,
COALECSE(MAX(m3_1), 0) + COALECSE(MAX(m3_2), 0) + COALECSE(MAX(m3_3), 0) + COALECSE(MAX(m3_4), 0) +
COALECSE(MAX(m3_5), 0) + COALECSE(MAX(m3_6), 0) + COALECSE(MAX(m3_7), 0) subtotal
FROM (
SELECT date, m3 m3_1, null m3_2, null m3_3, null m3_4, null m3_5, null m3_6, null m3_7 FROM XT1
UNION ALL
SELECT date, null, m3, null, null, null, null, null FROM XT2
UNION ALL
SELECT date, null, null, m3, null, null, null, null FROM XT3
UNION ALL
............................................................
SELECT date, null, null, null, null, null, null, m3 FROM XT7
) t
GROUP BY date
I have a table called results with 5 columns.
I'd like to use the title column to find rows that are say: WHERE title like '%for sale%' and then listing the most popular words in that column. One would be for and another would be sale but I want to see what other words correlate with this.
Sample data:
title
cheap cars for sale
house for sale
cats and dogs for sale
iphones and androids for sale
cheap phones for sale
house furniture for sale
Results (single words):
for 6
sale 6
cheap 2
and 2
house 2
furniture 1
cars 1
etc...
You can extract words with some string manipulation. Assuming you have a numbers table and that words are separated by single spaces:
select substring_index(substring_index(r.title, ' ', n.n), ' ', -1) as word,
count(*)
from results r join
numbers n
on n.n <= length(title) - length(replace(title, ' ', '')) + 1
group by word;
If you don't have a numbers table, you can construct one manually using a subquery:
from results r join
(select 1 as n union all select 2 union all select 3 union all . . .
) n
. . .
The SQL Fiddle (courtesy of #GrzegorzAdamKowalski) is here.
You can use ExtractValue in some interesting way. See SQL fiddle here: http://sqlfiddle.com/#!9/0b0a0/45
We need only one table:
CREATE TABLE text (`title` varchar(29));
INSERT INTO text (`title`)
VALUES
('cheap cars for sale'),
('house for sale'),
('cats and dogs for sale'),
('iphones and androids for sale'),
('cheap phones for sale'),
('house furniture for sale')
;
Now we construct series of selects which extract whole words from text converted to XML. Each select extracts N-th word from the text.
select words.word, count(*) as `count` from
(select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[1]') as word from `text`
union all
select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[2]') from `text`
union all
select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[3]') from `text`
union all
select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[4]') from `text`
union all
select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[5]') from `text`) as words
where length(words.word) > 0
group by words.word
order by `count` desc, words.word asc
This would give you single words (Just if I understand what your single word means.):
select concat(val,' ',cnt) as result from(
select (substring_index(substring_index(t.title, ' ', n.n), ' ', -1)) val,count(*) as cnt
from result t cross join(
select a.n + b.n * 10 + 1 n
from
(select 0 as n union all select 1 union all select 2 union all select 3
union all select 4 union all select 5 union all select 6
union all select 7 union all select 8 union all select 9) a,
(select 0 as n union all select 1 union all select 2 union all select 3
union all select 4 union all select 5 union all select 6
union all select 7 union all select 8 union all select 9) b
order by n
) n
where n.n <= 1 + (length(t.title) - length(replace(t.title, ' ', '')))
group by val
order by cnt desc
) as x
Result should be looks like this :
Result
--------
for 6
sale 6
house 2
and 2
cheap 2
phones 1
iphones 1
dogs 1
furniture 1
cars 1
androids 1
cats 1
But if the single word you need like this :
result
-----------
for 6 sale 6 house 2 and 2 cheap 2 phones 1 iphones 1 dogs 1 furniture 1 cars 1 androids 1 cats 1
Just modify the query above to:
select group_concat(concat(val,' ',cnt) separator ' ') as result from( ...
Update
Idea taken from https://stackoverflow.com/a/17942691/98491
This query works on my machine (MySQL 5.7), however Sqlfiddle reports an error.
The basic idea is that you should either create a table with numbers from 1 to maximum word occurence (like 4) in your field or as I did, use a UNION 1 .. 4 for simplicity.
CREATE TABLE products (
`id` int,
`name` varchar(45)
);
INSERT INTO products
(`id`, `name`)
VALUES
(1, 'for sale'),
(2, 'for me'),
(3, 'for you'),
(4, 'you and me')
;
SELECT name, COUNT(*) as count FROM
(
SELECT
product.id,
SUBSTRING_INDEX(SUBSTRING_INDEX(product.name, ' ', numbers.n), ' ', -1) name
FROM
(
SELECT 1 AS n
UNION SELECT 2
UNION SELECT 3
UNION SELECT 4
) AS numbers
INNER JOIN products product
ON CHAR_LENGTH(product.name)
-CHAR_LENGTH(REPLACE(product.name, ' ', ''))>=numbers.n-1
ORDER BY
id, n
)
AS result
GROUP BY name
ORDER BY count DESC
Result will be
for | 3
you | 2
me | 2
and | 1
sale| 1
SQL is not well suited for this task, While possible there are limitations (the number of words for example)
a quick PHP script to do the same task may be easier to use long term (and likely quicker too)
<?php
$rows = [
"cheap cars for sale",
"house for sale",
"cats and dogs for sale",
"iphones and androids for sale",
"cheap phones for sale",
"house furniture for sale",
];
//rows here should be replaced by the SQL result
$wordTotals = [];
foreach ($rows as $row) {
$words = explode(" ", $row);
foreach ($words as $word) {
if (isset($wordTotals[$word])) {
$wordTotals[$word]++;
continue;
}
$wordTotals[$word] = 1;
}
}
arsort($wordTotals);
foreach($wordTotals as $word => $count) {
echo $word . " " . $count . PHP_EOL;
}
Output
for 6
sale 6
and 2
cheap 2
house 2
phones 1
androids 1
furniture 1
cats 1
cars 1
dogs 1
iphones 1
Here is working SQL Fiddle: http://sqlfiddle.com/#!9/0b0a0/32
Let's start with two tables - one for texts and one for numbers:
CREATE TABLE text (`title` varchar(29));
INSERT INTO text
(`title`)
VALUES
('cheap cars for sale'),
('house for sale'),
('cats and dogs for sale'),
('iphones and androids for sale'),
('cheap phones for sale'),
('house furniture for sale')
;
CREATE TABLE iterator (`index` int);
INSERT INTO iterator
(`index`)
VALUES
(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13),(14),(15),
(16),(17),(18),(19),(20),(21),(22),(23),(24),(25),(26),(27),(28),(29),(30)
;
The second table, iterator must contains numbers from 1 to N where N higher or equal to the lenght of the longest string in text.
Then, run this query:
select
words.word, count(*) as `count`
from
(select
substring(concat(' ', t.title, ' '), i.index+1, j.index-i.index) as word
from
text as t, iterator as i, iterator as j
where
substring(concat(' ', t.title), i.index, 1) = ' '
and substring(concat(t.title, ' '), j.index, 1) = ' '
and i.index < j.index
) AS words
where
length(words.word) > 0
and words.word not like '% %'
group by words.word
order by `count` desc, words.word asc
There are two selects. Outer one simply groups and counts single words (words of length greater than 0 and without any spaces). Inner one extracts all strings starting from any space character and ending with any other space character, so strings aren't words (despite naming this subquery words) because they can contain other spaces than starting and ending one.
Results:
word count
for 6
sale 6
and 2
cheap 2
house 2
androids 1
cars 1
cats 1
dogs 1
furniture 1
iphones 1
phones 1
I have the following table stops how can I check whether the following stops name order GHI, JKL, MNO is available in my stops table?
stops table:
CREATE TABLE IF NOT EXISTS stops
(
stop_id INT(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,
name varchar(30) NOT NULL,
lat double(10,6),
longi double(10,6)
);
Simple:
1 ABC
2 DEF
3 GHI
4 JKL
5 MNO
6 PQR
7 SDU
8 VWX
This query will return 1 when there is an ordered of 'GHI','JKL','MNO':
SELECT 1
FROM stops s1
JOIN stops s2 ON s1.stop_id = s2.stop_id - 1
JOIN stops s3 ON s2.stop_id = s3.stop_id - 1
WHERE CONCAT(s1.name, s2.name, s3.name) = CONCAT('GHI','JKL','MNO')
SQL Fiddle Demo
This is a variation of the well known "find equal sets" task.
You need to insert the searched route into a table with a sequenced stop_id:
create table my_stops(stop_id INT NOT NULL,
name varchar(30) NOT NULL);
insert into my_stops (stop_id, name)
values (1, 'GHI'),(2, 'JKL'),(3, 'MNO');
Then you join and calculate the difference between both sequences. This returns a totally meaningless number, but always the same for consecutive values:
select s.*, s.stop_id - ms.stop_id
from stops as s join my_stops as ms
on s.name = ms.name
order by s.stop_id;
Now group by that meaningless number and search for a count equal to the number of searched steps:
select min(s.stop_id), max(s.stop_id)
from stops as s join my_stops as ms
on s.name = ms.name
group by s.stop_id - ms.stop_id
having count(*) = (select count(*) from my_stops)
See Fiddle
Another alternative:
select 1
from stops x
where x.name = 'GHI'
and (select GROUP_CONCAT(name order by y.stop_id)
from stops y where y.stop_id between x.stop_id + 1
and x.stop_id + 2
) = 'JKL,MNO';
I have seen this issue in SF, but me being a noob I just can't get my fried brain around them. So please forgive me if this feels like repetition.
My Sample Table
--------------------------
ID | Supplier | QTY
--------------------------
1 1 2
2 1 2
3 2 5
4 3 2
5 1 3
6 2 4
I need to get the rows "UNTIL" the cumulative total for "QTY" is equal or greater than 5 in descending order for a particular supplier id.
In this example, for supplier 1, it will be rows with the ids of 5 and 2.
Id - unique primary key
Supplier - foreign key, there is another table for supplier info.
Qty - double
It ain't pretty, but I think this does it and maybe it can be the basis of something less cumbersome. Note that I use a "fake" INNER JOIN just to get some variable initialized for the first time--it serves no other role.
SELECT ID,
supplier,
qty,
cumulative_qty
FROM
(
SELECT
ID,
supplier,
qty,
-- next line keeps a running total quantity by supplier id
#cumulative_quantity := if (#sup <> supplier, qty, #cumulative_quantity + qty) as cumulative_qty,
-- next is 0 for running total < 5 by supplier, 1 the first time >= 5, and ++ after
#reached_five := if (#cumulative_quantity < 5, 0, if (#sup <> supplier, 1, #reached_five + 1)) as reached_five,
-- next takes note of changes in supplier being processed
#sup := if(#sup <> supplier, supplier, #sup) as sup
FROM
(
--this subquery is key for getting things in supplier order, by descending id
SELECT *
FROM `sample_table`
ORDER BY supplier, ID DESC
) reverse_order_by_id
INNER JOIN
(
-- initialize the variables used to their first ever values
SELECT #cumulative_quantity := 0, #sup := 0, #reached_five := 0
) only_here_to_initialize_variables
) t_alias
where reached_five <= 1 -- only get things up through the time we first get to 5 or above.
How about this? Using two variables.
SQLFIDDLE DEMO
Query:
set #tot:=0;
set #sup:=0;
select x.id, x.supplier, x.ctot
from (
select id, supplier, qty,
#tot:= (case when #sup = supplier then
#tot + qty else qty end) as ctot,
#sup:=supplier
from demo
order by supplier asc, id desc) x
where x.ctot >=5
;
| ID | SUPPLIER | CTOT |
------------------------
| 2 | 1 | 5 |
| 1 | 1 | 7 |
| 3 | 2 | 5 |
Standard SQL has no concept of 'what row number am I up to', so this can only be implemented using something called a cursor. Writing code with cursors is something like writing code with for loops in other languages.
An example of how to use cursors is here:
http://dev.mysql.com/doc/refman/5.0/en/cursors.html
Here is a rough demo about cursor, may be it's helpful.
CREATE TABLE #t
(
ID INT IDENTITY,
Supplier INT,
QTY INT
);
TRUNCATE TABLE #t;
INSERT INTO #t (Supplier, QTY)
VALUES (1, 2),
(1, 2),
(2, 5),
(3, 2),
(1, 3);
DECLARE #sum AS INT;
DECLARE #qty AS INT;
DECLARE #totalRows AS INT;
DECLARE curSelectQTY CURSOR
FOR SELECT QTY
FROM #t
ORDER BY QTY DESC;
OPEN curSelectQTY;
SET #sum = 0;
SET #totalRows = 0;
FETCH NEXT FROM curSelectQTY INTO #qty;
WHILE ##FETCH_STATUS = 0
BEGIN
SET #sum = #sum + #qty;
SET #totalRows = #totalRows + 1;
IF #sum >= 5
BREAK;
END
SELECT TOP (#totalRows) *
FROM #t
ORDER BY QTY DESC;
CLOSE curSelectQTY;
DEALLOCATE curSelectQTY;
SELECT x.*
FROM supplier_stock x
JOIN supplier_stock y
ON y.supplier = x.supplier
AND y.id >= x.id
GROUP
BY supplier
, id
HAVING SUM(y.qty) <=5;
I have a stored Procedure that I cant get to work:
ALTER PROCEDURE GetBrands
#sortColumn INT
AS
SELECT DISTINCT(tblBrand.BrandID),
tblBrandinCategory.CategoryID,
tblBrand.BrandName AS Brand,
AVG(tblReview.Grade) AS AverageGrade,
COUNT(tblReview.ReviewID) AS Review
FROM tblBrand LEFT JOIN
tblBrandinCategory ON tblBrand.BrandID = tblBrandinCategory.BrandID LEFT JOIN
tblReview ON tblBrand.BrandID = tblReview.BrandID
GROUP BY tblBrand.BrandID, tblBrandinCategory.CategoryID, tblBrand.BrandName
ORDER BY
CASE
WHEN #sortColumn = 1 THEN Brand
WHEN #sortColumn = 2 THEN Review
WHEN #sortColumn = 4 THEN AverageGrade
ELSE Brand
END
The result that I want to have is a list with brands, that only will be displayed once. The tblBrandInCategory messes that up for me.
tblBrand
BrandID BrandName
1 Nike
2 Adidas
3 Puma
tblCategory
CategoryID CategoryName
1 Shoes
2 Shorts
3 Pants
tblBrandInCategory
CategoryID BrandID
1 1
2 1
3 1
tblReview
ReviewID Grade BrandID
1 5 1
2 9 1
3 2 1
I get the result multiplyed with three becouse BrandID 1 exist 3 times in tblBrandInCategoiry.
Another problem is that in the ORDER BY CASE I get errors that AverageGrade is not recognizable, but tblReview.Grade is fine, but I want to order it on the Average Grade.
Another problem is that in the ORDER BY CASE I get errors that
AverageGrade is not recognizable, but tblReview.Grade is fine, but I
want to order it on the Average Grade.
AverageGrade can not be used as a column to sort until your main query is the Sub Query....
I get the result multiplyed with three becouse BrandID 1 exist 3 times
in tblBrandInCategoiry.
create table #tblBrand
(
BrandID int,
BrandName varchar(10)
)
create table #tblCategory
(
CategoryID int,
CategoryName varchar(10)
)
create table #tblBrandInCategory
(
CategoryID int,
BrandID int
)
create table #tblReview
(
ReviewID int,
Grade int,
BrandID int
)
insert into #tblBrand(BrandID, BrandName)values(1, 'Nike')
insert into #tblBrand(BrandID, BrandName)values(2, 'Adidas')
insert into #tblBrand(BrandID, BrandName)values(3, 'Puma')
insert into #tblCategory(CategoryID, CategoryName)values(1, 'Shoes')
insert into #tblCategory(CategoryID, CategoryName)values(2, 'Shorts')
insert into #tblCategory(CategoryID, CategoryName)values(3, 'Pants')
insert into #tblBrandInCategory(CategoryID, BrandID)values(1, 1)
insert into #tblBrandInCategory(CategoryID, BrandID)values(2, 1)
insert into #tblBrandInCategory(CategoryID, BrandID)values(3, 1)
insert into #tblReview(ReviewID, Grade, BrandID)values(1, 5, 1)
insert into #tblReview(ReviewID, Grade, BrandID)values(2, 5, 9)
insert into #tblReview(ReviewID, Grade, BrandID)values(3, 2, 1)
Select BrandID, Brand, AverageGrade, Review
From
(
SELECT DISTINCT(#tblBrand.BrandID),
--#tblBrandinCategory.CategoryID,
#tblBrand.BrandName AS Brand,
AVG(#tblReview.Grade) AS AverageGrade,
COUNT(#tblReview.ReviewID) AS Review
FROM #tblBrand
LEFT JOIN #tblBrandinCategory ON #tblBrand.BrandID = #tblBrandinCategory.BrandID
LEFT JOIN #tblReview ON #tblBrand.BrandID = #tblReview.BrandID
GROUP BY #tblBrand.BrandID, #tblBrandinCategory.CategoryID, #tblBrand.BrandName
)K
ORDER BY
CASE
WHEN #sortColumn = 1 THEN Brand
WHEN #sortColumn = 2 THEN Review
WHEN #sortColumn = 4 THEN AverageGrade
ELSE Brand
drop table #tblBrand
drop table #tblCategory
drop table #tblBrandInCategory
drop table #tblReview
Final Resultset
Just edit the SELECT to leave off tblBrandInCategory results:
SELECT tblBrand.BrandID,
tblBrand.BrandName AS Brand,
AVG(tblReview.Grade) AS AverageGrade,
COUNT(tblReview.ReviewID) AS Review
FROM tblBrand
--tblBrandinCategory isn't even needed for what you're doing
--LEFT JOIN tblBrandinCategory ON tblBrand.BrandID = tblBrandinCategory.BrandID
LEFT JOIN tblReview ON tblBrand.BrandID = tblReview.BrandID
GROUP BY tblBrand.BrandID, tblBrand.BrandName
This would need to be edited further to account for null joins, where a grade does not exist (maybe with AVG(ISNULL(tblReview.Grade, 0))). Depends on your requirements.