This question in regarding mySQL.
I am reading data from an .csv file that looks like this
original table
------------------------------------
Id 2018M01 2018M02 2018M03
------------------------------------
EMEA 3 1 4
ASPAC 4 5 4
ASPAC 1 2 1
expected result
---------------------
ID Month Qty
---------------------
EMEA 2018M01 3
EMEA 2018M02 1
EMEA 2018M03 4
ASPAC 2018M01 4
.......
The months column header are dynamic, that is each month there will be new months and old months will be removed. However the total number of columns will remain the same.
Hence whenever the month columns headers change I would like the SQL code to dynamically read and provide correct results without me having to manually change several parts of the code.
I have written the following code; Code is to unpivot the month columns. However I tested the code by manually making changes to the .csv file headers by changing 2018M03 to 2018M04, and rerunning the SQL code, but it still seems to print the old data . What am I doing wrong ?
Thank you. I am fairly new to SQL.
DROP TABLE IF EXISTS book;
CREATE TABLE book (
ID VARCHARACTER(10),
2018M01 decimal(4,2),
2018M02 decimal(4,2),
2018M03 decimal(4,2)
);
LOAD DATA LOCAL INFILE '/Users/blytonpereira/Desktop/Book1.csv' REPLACE INTO TABLE book
FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 LINES;
DESCRIBE book;
SELECT ID, '2018M01' AS month, 2018M01 AS qty from book
UNION ALL
SELECT ID, '2018M02' AS month, 2018M02 AS qty from book
UNION ALL
SELECT ID, '2018M03' AS month, 2018M03 AS qty from book;
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'select ID, ''',
c.column_name,
''' AS month, ',
c.column_name,
' as qty
from book
where ',
c.column_name,
' > 0'
) SEPARATOR ' UNION ALL '
) INTO #sql
FROM information_schema.columns c
where c.table_name = 'book'
and c.column_name not in ('id')
order by c.ordinal_position;
SET #sql
= CONCAT('select id, month, qty
from
(', #sql, ') x order by id');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
);
** Updated answer:
This solution is dynamic, whatever columns you have in table, it will populate them and extract information in required format. For sure columns have to have something in common, like they're all begins with "2018", you can change that as needed in Query.
SELECT
GROUP_CONCAT(
CONCAT(
'SELECT id, ''', COLUMN_NAME, ''' as month, ', COLUMN_NAME, ' as QTY FROM t1 ') SEPARATOR ' UNION ALL ')
FROM
`INFORMATION_SCHEMA`.`COLUMNS`
WHERE
`COLUMN_NAME` LIKE '2018%'
INTO #sql;
SET #query = CONCAT('select id, month, QTY from (' , #sql , ') x order by id;');
SELECT #query;
PREPARE stmt FROM #query;
EXECUTE stmt;
**Note: the query has 2 outputs, first is the prepared concatenated query string (just to know what it looks like before run), and the other is the actual data. If you want only actual data you can comment (SELECT #query;) or remove it.
Related
I already went through the details in the link (Mysql Convert Column to row (Pivot table )). As the number of Columns is high and using union all on all of them would be time taking. I decided to use the last resolution in the given link. I was able to run the query the results were:
The issue is the acct getting included as data and also I want to create a table from the result . So Can these entries be excluded and how can I create a table from the results? (new to SQL)
The Code:
SET SESSION group_concat_max_len = 92160;
SET #target_schema='rd';
SET #target_table='pbc_gl';
SET #target_where='`acct`';
SELECT
GROUP_CONCAT(qry SEPARATOR ' UNION ALL ')
INTO #sql
FROM (
SELECT
CONCAT('SELECT `acct`,', QUOTE(COLUMN_NAME), ' AS `Business_Unit`,`', COLUMN_NAME, '` AS `value` FROM `', #target_table, '` WHERE ', #target_where) qry
FROM (
SELECT `COLUMN_NAME`
FROM `INFORMATION_SCHEMA`.`COLUMNS`
WHERE `TABLE_SCHEMA`=#target_schema
AND `TABLE_NAME`=#target_table
) AS `A`
) AS `B` ;
PREPARE s FROM #sql;
EXECUTE s;
DEALLOCATE PREPARE s;
The Question:
How do I limit the number of columns displayed/produced on a MYSQL pivot table?
My Setup:
I have a table named "updates" that looks like the following:
I have the following snippet of query (This is only part of the query, the whole thing only adds more columns from other tables but this is the only section that gets pivoted):
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'MAX(IF(Date = ''',
Date,
''', Description, NULL)) AS ',
CONCAT("'",Date_Format(Date, '%d/%m/%Y'),"'")
)
)INTO #sql
FROM updates;
SET #sql = CONCAT('SELECT Action, ', #sql, ' FROM updates GROUP BY Action');
PREPARE stmt FROM #sql;
EXECUTE stmt;
The result of this query is as follows:
As you can see, this pivots the table as intended with the dates as columns. However, there is potential for these updates (to actions) to become very long before they are "closed" and not displayed. Therefore, I would like to limit the outcome to the latest 3 updates. BUT..Not per action as this would potentially still give me a lot of updates in the pivot table.
I would like to have the most recent 3 dates from the updates table with all updates for each date keeping this pivot format.
Example: The outcome table above would look the same but with the exception of the columns titled "02/10/2016" and "04/10/2016".
Thanks in advance for any assistance or advise.
For anyone else trying to solve this issue, I managed to use the following query to produce the desired results:
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'MAX(IF(Date = ''',
Date,
''', Description, NULL)) AS ',
CONCAT("'",Date_Format(Date, '%d/%m/%Y'),"'")
) ORDER BY Date ASC
) INTO #sql
FROM (
SELECT * FROM updates
GROUP BY Date
ORDER BY Date DESC
LIMIT 2)
AS updates;
SET #sql = CONCAT('SELECT Action, ', #sql, ' FROM updates GROUP BY Action');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
I have a table with two columns, following is the schema:
create table scan( `application_name` varchar(255) NOT NULL, `defect_type` varchar(255) NOT NULL);
And the data is populated accordingly. The table stores data for "Application" and its corresponding "Defect Type". I want to perform following 2 actions on this table:
Get the Top 3 "Defect Type" of a particular application in terms of percent.
Transpose the output from above where the values in "Defect Type" (defect_type) become the columns and its corresponding percent (finalPercent) as the its value.
I am able to achieve 1, following is the SQL Fiddle:
SQLFiddle
However, I am not able to transpose the output as per the requirement. Ideally there should be a single row as follows after both 1 & 2 together:
application_name | CrossSide | CSS | XML
A | 33.33 | 33.33 | 16.67
Thank you!
You can build a dynamic pivot query with group_concat and then execute it:
set #sql = null;
set #total = (select count(*) as totalcount from scan where application_name = "a");
select group_concat(distinct
concat(
'round(((count(case when defect_type = ''',
defect_type,
''' then application_name end)/ #total ) * 100 ), 2) as `',
defect_type, '`'
)
) into #sql
from ( select defect_type
from scan
where application_name = "a"
group by defect_type
order by count(defect_type) desc
limit 3) t;
set #sql = concat(
'select application_name, ', #sql,
' from scan
where application_name = "a"
group by application_name'
);
prepare stmt from #sql;
execute stmt;
deallocate prepare stmt;
SQLFiddle
I am trying to turn this statement:
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'max(case when year = ',
year,
' then experience_rate end) AS `',
year, '-Pen`'
) ORDER BY year
) INTO #sql
FROM
spooner_pec;
SET #sql = CONCAT('SELECT policy_number, primary_name, ', #sql, '
FROM spooner_pec
GROUP BY policy_number');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
Into something like this:
SELECT policy_number, primary_name, (SELECT GROUP_CONCAT(DISTINCT CONCAT('max(case when year = ', year, ' then experience_rate end) AS `', year, '-Pen`') ORDER BY year))
FROM spooner_pec
GROUP BY policy_number
But as you can see by the fiddle, I am getting some strange output as a column instead of the actual columns, what am I doing wrong here?
SQLFiddle
I generally do regular expression search to get all the code in single line or selected code in single line.
Use find & replace
1. Under find section type "\n" --find all next line keywords
2. Under replace section type " " -- replace \n with one blank space
3. Tick "Use Regular Expression"
Your result will be in single line.
Before
After
Replace with section is a blank space
*Note: This approach works on Windows platform and most of the editors.
How can I convert dynamically a key-value table using Case without specifying row names?
For example I have this table named key_value:
id key value
1 name john
2 fname akbar
3 jobs Software enginer
. . .
. . .
. . .
. . .
. . .
n n n
I want to convert all these rows dynamically to columns without specifying key name like:
name fname jobs............................n
john akbar sofware engineer...........n
I have used:
Max(Case WHEN key='name' THEN value END) AS name
In this query I know my key.
What if I don't know my fields and I don't know how many fields I have?
I want to convert all of this dynamically without specifying my fields.
This was also my question way back
SQL Query fields as columns
I modified it to answer yours and I hope it helps just like it did to me!
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'MAX(IF(a.xvalue = ''',
xvalue,
''', a.xvalue, NULL)) AS ',
xkey
)
) INTO #sql
FROM key_value;
SET #sql = CONCAT('SELECT ', #sql, '
FROM key_value a
LEFT JOIN key_value AS b
ON a.id=b.id');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SQL Fiddle