Retrieve Distinct concat values from MySQL table - mysql

I have an SQL table advert
id name cat
11 abc ab
12 acb ab, bc
13 abb bcd
14 abcd ad
15 acbd de
16 abbd ad
On using DISTINCT function I am getting an output like this
Query:
SELECT DISTINCT cat FROM advert;
Output:
ab
ab, bc
bcd
ad
de
WHAT changes do I need to make in my query for output like this
ab
bc
bcd
ad
de

select distinct trim(substring_index(substring_index(cat,',',n),',',-1)) as cat
from t join (select 1 as n union all select 2 union all select 3) r
on cat like concat('%',repeat(',%',n-1))

I think you should change your table structure and make it like this.
tblName
id | name
11 abc
12 acb
13 abb
14 abcd
15 acbd
16 abbd
tblCat
id | name_id | cat
some ids* 11 ab
12 ab
12 bc
13 bcd
14 ad
15 de
16 ad
In this way you can easily query and manage your data in your tables.

You should fix your data structure so you are not storing comma-delimited lists in columns. That is the wrong way to store data in a relational database . . . as you can see by the problems for answering this simple question. What you want is a junction table.
Sometimes, we are stuck with other peoples bad designs. You say that there are only two or values, then you can do:
select cat
from ((select substring_index(cat, ', ', 1) as cat
from advert
) union all
(select substring_index(substring_index(cat, ', ', 2), ', ', -1) as cat
from advert
where cat like '%, %'
) union all
(select substring_index(substring_index(cat, ', ', 3), ', ', -1) as cat
from advert
where cat like '%, %, %'
)
) c
group by cat;

First... I would create a statement that would turn all the rows into one big massive comma delimited list.
DECLARE #tmp VarChar(max)
SET #tmp = ''
SELECT #tmp = #tmp + ColumnA + ',' FROM TableA
Then use the table valued udf split described by this SO article to turn that massive string back into a table with a distinct clause to ensure that it's unique.
https://stackoverflow.com/a/2837662/261997
SELECT DISTINCT * FROM dbo.Split(',', #tmp)
Full code example:
if object_id('dbo.Split') is not null
drop function dbo.Split
go
CREATE FUNCTION dbo.Split (#sep char(1), #s varchar(512))
RETURNS table
AS
RETURN (
WITH Pieces(pn, start, stop) AS (
SELECT 1, 1, CHARINDEX(#sep, #s)
UNION ALL
SELECT pn + 1, stop + 1, CHARINDEX(#sep, #s, stop + 1)
FROM Pieces
WHERE stop > 0
)
SELECT pn,
SUBSTRING(#s, start, CASE WHEN stop > 0 THEN stop-start ELSE 512 END) AS s
FROM Pieces
)
go
declare #t table (colA varchar(max))
insert #t select '111, 223'
union all select '333'
union all select '444'
union all select '777,999';
select ltrim(rtrim(s.s)) as colC
from #t t
cross apply
dbo.split(',', t.colA) s

Related

How to find with partial string from comma separated list of string then need to remove entire string from the list of values

We have requirement in Mysql query, to find with partial string from list of comma separated string. Then need to remove a found a string from comma separated list.
As per the below example, we need to find a string starting with "Pending" then need to remove found string from the comma separated list through Mysql query.
|-------------------|-----------------------------------------|
| Id | Tag |
|-------------------|-----------------------------------------|
| 1 | Completed, #4CHD, Pending with ABC |
|-------------------|-----------------------------------------|
| 2 | Open, Pending with Mrg, #4CHD |
|-------------------|-----------------------------------------|
| 3 | Pending with cons, Resolved |
|-------------------|-----------------------------------------|
Output should be:
|-------------------|-----------------------|
| Id | Tag |
|-------------------|-----------------------|
| 1 | Completed, #4CHD |
|-------------------|-----------------------|
| 2 | Open, #4CHD |
|-------------------|-----------------------|
| 3 | Resolved |
|-------------------|-----------------------|
For MySQL 8+
SELECT test.id, GROUP_CONCAT(TRIM(jsontable.value)) Tag
FROM test
CROSS JOIN JSON_TABLE(CONCAT('["', REPLACE(test.Tag, ',', '","'), '"]'),
'$[*]' COLUMNS( value VARCHAR(255) PATH '$')
) jsontable
WHERE TRIM(jsontable.value) NOT LIKE CONCAT(#criteria, '%')
GROUP BY test.id
For MySQL 5.x
SELECT test.id,
GROUP_CONCAT(TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(test.Tag, ',', nums.num), ',', -1))) Tag
FROM test
JOIN (SELECT 1 num UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5) nums
ON nums.num <= 1 + LENGTH(test.Tag) - LENGTH(REPLACE(test.Tag, ',', ''))
WHERE TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(test.Tag, ',', nums.num), ',', -1)) NOT LIKE CONCAT(#criteria, '%')
GROUP BY test.id
fiddle
No rows should be ingnored in output. Let us assume 3rd row has Tag value as "Pending with cons" alone. I this case first two is getting displayed in output and 3rd row is ignored. My requirement is 3rd also should be displayed with empty Tag.
If so LEFT JOIN (and moving condition expression from WHERE to ON) needed:
SELECT test.id,
GROUP_CONCAT(TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(test.Tag, ',', nums.num), ',', -1))) Tag
FROM test
LEFT JOIN (SELECT 1 num UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5) nums
ON nums.num <= 1 + LENGTH(test.Tag) - LENGTH(REPLACE(test.Tag, ',', ''))
AND TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(test.Tag, ',', nums.num), ',', -1)) NOT LIKE CONCAT(#criteria, '%')
GROUP BY test.id
fiddle

How to get top x values from a list of a sum ( 3 tables ) - MySQL

Sorry for the title but I didn't know what can I put in it :(.
I have 3 tables : team , nation and results.
Table team contains -> id, name, id_nation
Table nation contains -> id, name
Table results contains -> id, id_team, points
My problem is : I want to display the list of points by nations BUT, if there are 7 teams from a same nation, I have to "get" only the top 5 scores of this teams to make the total score of this nation.
Example, I have 7 teams ( team 1 , team 2 , team 3, ... team 7 ) from Spain who has id_nation like the id of the Spain nation.
For each teams the sum of the total points from results table are like that :
Team 1 => 7 points
Team 2 => 9 points
Team 3 => 5 points
Team 4 => 5 points
Team 5 => 1 point
Team 6 => 10 points
Team 7 => 9 points
I remember to you that all these 7 teams are from Spain. So, normally, the total of Spain points must be -> ( 7 + 9 + 5 + 5 + 1 + 10 + 9 ) = 46 points.
BUT, me, like I said, I want only the top 5 teams by country. So, I want, for Spain in this example, a total of ( 10 + 9 + 9 + 7 + 5 ) = 40 points ans not 46 ( for the Spain team ).
I need your help because I don't know how to have this result in SQL ( MySQL ).
Thanks a lot in advance
This is a pain. Basically, you need variables to enumerate the rows and then filtering and aggregation:
select id_nation, sum(points)
from (select r.*,
(#rn := if(#n = r.id_nation, #rn + 1,
if(#n := r.id_nation, 1, 1)
)
) as rn
from (select r.*, t.id_nation
from results r join
teams t
on r.team_id = t.id
order by t.id_nation, r.points desc
) r cross join
(select #n := -1, #rn := 0) params
) r
where rn <= 5
group by id_nation;
Here is a sample to make it on mysql 5.7.23,
The better way is using a window function, but unfortunately, it only supports on mysql 8.0 or higher than it.
So you can try to declare a variable. the make row number then filter you want to SUM top of row.
MySQL 5.6 Schema Setup:
CREATE TABLE Result(
points int
);
INSERT INTO Result VALUES (7);
INSERT INTO Result VALUES (9);
INSERT INTO Result VALUES (5);
INSERT INTO Result VALUES (5);
INSERT INTO Result VALUES (1);
INSERT INTO Result VALUES (10);
INSERT INTO Result VALUES (9);
Query 1:
SELECT SUM(points)
FROM (
SELECT t.points,(#rn := #rn + 1) rn
FROM result t CROSS JOIN (SELECT #rn := 0 rn) v
ORDER BY points DESC
) t
where rn <= 5
Results:
| SUM(points) |
|-------------|
| 40 |

Split values to multiple row on MySQL

Here is my data in mysql table:
+---------+-------------------+------------+
| ID | Name | Class |
+---------+-------------------+------------+
| 1, 2, 3 | Alex, Brow, Chris | Aa, Bb, Cc |
+---------+-------------------+------------+
I want split values to multiple rows to get data as the below format.
1 Alex Aa
2 Brow Bb
3 Chris Cc
How can I do that?
One trick is to join to a Tally table with numbers.
Then use SUBSTRING_INDEX to get the parts.
If you don't already have a numbers table, here's one way.
drop table if exists Digits;
create table Digits (n int primary key not null);
insert into Digits (n) values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
drop table if exists Nums;
create table Nums (n int primary key not null);
insert into Nums (n)
select (n3.n*100+n2.n*10+n1.n) as n
from Digits n1
cross join Digits n2
cross join Digits n3;
Then it can be used to unfold those columns
Sample data:
drop table if exists YourTable;
create table YourTable (
ID varchar(30) not null,
Name varchar(30) not null,
Class varchar(30) not null
);
insert into YourTable
(ID, Name, Class) values
('1, 2, 3', 'Alex, Brow, Chris', 'Aa, Bb, Cc')
, ('4, 5, 6', 'Drake, Evy, Fiona', 'Dd, Ee, Ff')
;
The query:
SELECT
LTRIM(SUBSTRING_INDEX( SUBSTRING_INDEX( t.ID, ',', Nums.n), ',', -1)) AS Id,
LTRIM(SUBSTRING_INDEX( SUBSTRING_INDEX( t.Name, ',', Nums.n), ',', -1)) AS Name,
LTRIM(SUBSTRING_INDEX( SUBSTRING_INDEX( t.Class, ',', Nums.n), ',', -1)) AS Class
FROM YourTable t
LEFT JOIN Nums ON n BETWEEN 1 AND (LENGTH(ID)-LENGTH(REPLACE(ID, ',', ''))+1);
Result:
Id Name Class
1 Alex Aa
2 Brow Bb
3 Chris Cc
4 Drake Dd
5 Evy Ee
6 Fiona Ff
Multiple values are not recommended for a one field but still if you want a solution you can split string by comma and insert to the table.
Please see this blog post which shows how to split https://nisalfdo.blogspot.com/2019/02/mysql-how-to-insert-values-from-comma.html#more

How to write an SQL query that expands a comma delimited field into multiple fields?

First of all...I know it's bad to have comma separated values in tables and no I'm not able to change it.
I have several tables that contain the following data:
************** Table 1 **********
stock_id products_id stock_attributes
5271 279 1559,2764
************** Table 2 *********************
products_attributes_id products_id options_id options_values_id
1559 279 2 8
2764 279 3 63
************** Table 3 ************************
products_options_id products_options_name
2 Size
3 Color
************** Table 4 *****************
products_options_values_id products_options_values_name
14 Pink
63 Mint
13 Black
8 S
9 M
10 L
11 XL
What I'd like to do is create a query to take the field stock_attributes in Table 1 and expand it using the information in the Tables 2, 3 & 4 so I end up with the following:
*********** Resulting Table **********
stock_id products_id opt1 opt2 opt3 opt4
5271 279 Size S Color Mint
I can do this programmatically after the fact but I'm curious if it can be done in a single SQL query. I've found similar questions and answer on how to select a particular value from a comma delimited field but nothing to do this. Any help is appreciated.
If the number of attributes are fixed. You can simply split the column into multiple like this.
SELECT stock_id, products_id,
PARSENAME(REPLACE(stock_attributes,',','.'),2) a1,
PARSENAME(REPLACE(stock_attributes,',','.'),1) a2
FROM Table1
To further expand and include the values from other tables, you can do simple JOIN
select pTable.stock_id, pTable.products_id,
(select products_options_name from Table3 where products_options_id = t2.options_id) as opt1,
(select products_options_values_name from Table4 where products_options_values_id = t2.options_values_id) as opt2,
(select products_options_name from Table3 where products_options_id = t3.options_id) as opt3,
(select products_options_values_name from Table4 where products_options_values_id = t3.options_values_id) as opt4
from
(
SELECT stock_id, products_id,
PARSENAME(REPLACE(stock_attributes,',','.'),2) a1,
PARSENAME(REPLACE(stock_attributes,',','.'),1) a2
FROM Table1
) pTable
join Table2 t2 on pTable.a1 = t2.products_attributes_id and pTable.products_id = t2.products_id
join Table2 t3 on pTable.a2 = t3.products_attributes_id and pTable.products_id = t3.products_id
This is very long. So I stop until the point where have the data ready for a Dynamic SQL Pivot
This can solve any number of atributes. Just save the output to a table o create a stored procedure to have the data for the dynamic pivot.
tmp: Split the comma-separated values into rows
product_attributes: is just a select to see the result from the recursive function. Can't be removed
product_details: join the attributes with the other tables to get their values
product_attribute_count: each attribute needs a row_number so you can create header later. Also I realize you didnt need the COUNT()
product_pre_pivot: create headers and value, for option_id and option_value_id
product_pivot: separated those pair header into a single column
You can test each step to see the result .... SELECT * FROM [STEP#]
SQL Fiddle Demo
Code:
;WITH tmp([stock_id], [products_id], products_attributes_id, data_r) AS
(
SELECT
[stock_id], [products_id],
LEFT([stock_attributes], Charindex(',', [stock_attributes] + ',') - 1),
STUFF([stock_attributes], 1, Charindex(',', [stock_attributes] + ','), '')
FROM
Table1
UNION ALL
SELECT
[stock_id], [products_id],
LEFT(data_r, Charindex(',', data_r + ',') - 1),
STUFF(data_r, 1, Charindex(',', data_r + ','), '')
FROM
tmp
WHERE
data_r > ''
), product_attributes AS
(
SELECT
[stock_id], [products_id],
[products_attributes_id]
FROM
tmp
), product_details AS
(
SELECT
pa.*,
t2.options_id, t2.options_values_id,
t3.[products_options_name],
t4.[products_options_values_name]
FROM
product_attributes pa
JOIN
Table2 t2 ON pa.[products_id] = t2.[products_id]
AND pa.[products_attributes_id] = t2.[products_attributes_id]
JOIN
Table3 t3 ON t2.[options_id] = t3.[products_options_id]
JOIN
Table4 t4 ON t2.[options_values_id] = t4.[products_options_values_id]
), product_attribute_count AS
(
SELECT
*,
row_number() over (PARTITION BY products_id
ORDER BY options_id) as rn,
count(*) over (partition by products_id) cnt
FROM
product_details
), product_pre_pivot AS
(
SELECT
stock_id, products_id,
'opt' + CAST(2*rn - 1 as varchar(max)) as header1,
products_options_name as d_value,
'opt' + CAST(2*rn as varchar(max)) as header2,
products_options_values_name as a_value
FROM
product_attribute_count
), product_pivot AS
(
SELECT
stock_id, products_id,
header1 header, d_value value
FROM
product_pre_pivot
UNION ALL
SELECT
stock_id, products_id,
header2 header, a_value value
FROM
product_pre_pivot
)
SELECT *
FROM product_pivot
ORDER BY header
OUTPUT
| stock_id | products_id | header | value |
|----------|-------------|--------|-------|
| 5271 | 279 | opt1 | Size |
| 5271 | 279 | opt2 | S |
| 5271 | 279 | opt3 | Color |
| 5271 | 279 | opt4 | Mint |

How to find most popular word occurrences in MySQL?

I have a table called results with 5 columns.
I'd like to use the title column to find rows that are say: WHERE title like '%for sale%' and then listing the most popular words in that column. One would be for and another would be sale but I want to see what other words correlate with this.
Sample data:
title
cheap cars for sale
house for sale
cats and dogs for sale
iphones and androids for sale
cheap phones for sale
house furniture for sale
Results (single words):
for 6
sale 6
cheap 2
and 2
house 2
furniture 1
cars 1
etc...
You can extract words with some string manipulation. Assuming you have a numbers table and that words are separated by single spaces:
select substring_index(substring_index(r.title, ' ', n.n), ' ', -1) as word,
count(*)
from results r join
numbers n
on n.n <= length(title) - length(replace(title, ' ', '')) + 1
group by word;
If you don't have a numbers table, you can construct one manually using a subquery:
from results r join
(select 1 as n union all select 2 union all select 3 union all . . .
) n
. . .
The SQL Fiddle (courtesy of #GrzegorzAdamKowalski) is here.
You can use ExtractValue in some interesting way. See SQL fiddle here: http://sqlfiddle.com/#!9/0b0a0/45
We need only one table:
CREATE TABLE text (`title` varchar(29));
INSERT INTO text (`title`)
VALUES
('cheap cars for sale'),
('house for sale'),
('cats and dogs for sale'),
('iphones and androids for sale'),
('cheap phones for sale'),
('house furniture for sale')
;
Now we construct series of selects which extract whole words from text converted to XML. Each select extracts N-th word from the text.
select words.word, count(*) as `count` from
(select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[1]') as word from `text`
union all
select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[2]') from `text`
union all
select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[3]') from `text`
union all
select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[4]') from `text`
union all
select ExtractValue(CONCAT('<w>', REPLACE(title, ' ', '</w><w>'), '</w>'), '//w[5]') from `text`) as words
where length(words.word) > 0
group by words.word
order by `count` desc, words.word asc
This would give you single words (Just if I understand what your single word means.):
select concat(val,' ',cnt) as result from(
select (substring_index(substring_index(t.title, ' ', n.n), ' ', -1)) val,count(*) as cnt
from result t cross join(
select a.n + b.n * 10 + 1 n
from
(select 0 as n union all select 1 union all select 2 union all select 3
union all select 4 union all select 5 union all select 6
union all select 7 union all select 8 union all select 9) a,
(select 0 as n union all select 1 union all select 2 union all select 3
union all select 4 union all select 5 union all select 6
union all select 7 union all select 8 union all select 9) b
order by n
) n
where n.n <= 1 + (length(t.title) - length(replace(t.title, ' ', '')))
group by val
order by cnt desc
) as x
Result should be looks like this :
Result
--------
for 6
sale 6
house 2
and 2
cheap 2
phones 1
iphones 1
dogs 1
furniture 1
cars 1
androids 1
cats 1
But if the single word you need like this :
result
-----------
for 6 sale 6 house 2 and 2 cheap 2 phones 1 iphones 1 dogs 1 furniture 1 cars 1 androids 1 cats 1
Just modify the query above to:
select group_concat(concat(val,' ',cnt) separator ' ') as result from( ...
Update
Idea taken from https://stackoverflow.com/a/17942691/98491
This query works on my machine (MySQL 5.7), however Sqlfiddle reports an error.
The basic idea is that you should either create a table with numbers from 1 to maximum word occurence (like 4) in your field or as I did, use a UNION 1 .. 4 for simplicity.
CREATE TABLE products (
`id` int,
`name` varchar(45)
);
INSERT INTO products
(`id`, `name`)
VALUES
(1, 'for sale'),
(2, 'for me'),
(3, 'for you'),
(4, 'you and me')
;
SELECT name, COUNT(*) as count FROM
(
SELECT
product.id,
SUBSTRING_INDEX(SUBSTRING_INDEX(product.name, ' ', numbers.n), ' ', -1) name
FROM
(
SELECT 1 AS n
UNION SELECT 2
UNION SELECT 3
UNION SELECT 4
) AS numbers
INNER JOIN products product
ON CHAR_LENGTH(product.name)
-CHAR_LENGTH(REPLACE(product.name, ' ', ''))>=numbers.n-1
ORDER BY
id, n
)
AS result
GROUP BY name
ORDER BY count DESC
Result will be
for | 3
you | 2
me | 2
and | 1
sale| 1
SQL is not well suited for this task, While possible there are limitations (the number of words for example)
a quick PHP script to do the same task may be easier to use long term (and likely quicker too)
<?php
$rows = [
"cheap cars for sale",
"house for sale",
"cats and dogs for sale",
"iphones and androids for sale",
"cheap phones for sale",
"house furniture for sale",
];
//rows here should be replaced by the SQL result
$wordTotals = [];
foreach ($rows as $row) {
$words = explode(" ", $row);
foreach ($words as $word) {
if (isset($wordTotals[$word])) {
$wordTotals[$word]++;
continue;
}
$wordTotals[$word] = 1;
}
}
arsort($wordTotals);
foreach($wordTotals as $word => $count) {
echo $word . " " . $count . PHP_EOL;
}
Output
for 6
sale 6
and 2
cheap 2
house 2
phones 1
androids 1
furniture 1
cats 1
cars 1
dogs 1
iphones 1
Here is working SQL Fiddle: http://sqlfiddle.com/#!9/0b0a0/32
Let's start with two tables - one for texts and one for numbers:
CREATE TABLE text (`title` varchar(29));
INSERT INTO text
(`title`)
VALUES
('cheap cars for sale'),
('house for sale'),
('cats and dogs for sale'),
('iphones and androids for sale'),
('cheap phones for sale'),
('house furniture for sale')
;
CREATE TABLE iterator (`index` int);
INSERT INTO iterator
(`index`)
VALUES
(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13),(14),(15),
(16),(17),(18),(19),(20),(21),(22),(23),(24),(25),(26),(27),(28),(29),(30)
;
The second table, iterator must contains numbers from 1 to N where N higher or equal to the lenght of the longest string in text.
Then, run this query:
select
words.word, count(*) as `count`
from
(select
substring(concat(' ', t.title, ' '), i.index+1, j.index-i.index) as word
from
text as t, iterator as i, iterator as j
where
substring(concat(' ', t.title), i.index, 1) = ' '
and substring(concat(t.title, ' '), j.index, 1) = ' '
and i.index < j.index
) AS words
where
length(words.word) > 0
and words.word not like '% %'
group by words.word
order by `count` desc, words.word asc
There are two selects. Outer one simply groups and counts single words (words of length greater than 0 and without any spaces). Inner one extracts all strings starting from any space character and ending with any other space character, so strings aren't words (despite naming this subquery words) because they can contain other spaces than starting and ending one.
Results:
word count
for 6
sale 6
and 2
cheap 2
house 2
androids 1
cars 1
cats 1
dogs 1
furniture 1
iphones 1
phones 1