This question already has an answer here:
MySQL pivot row into dynamic number of columns
(1 answer)
Closed 6 years ago.
I have a Google Big Query table with the following columns:
date | user | group | value
----------------------------
date1 | user1 | group1 | 10
----------------------------
date1 | user2 | group1 | 5
----------------------------
date2 | user1 | group1 | 20
----------------------------
date2 | user2 | group1 | 10
---------------------------
etc...
Now I want to convert this to this:
group | date1 | date2
----------------------
group1 | 15 | 30
So I want to have the sum of value for each day per group. I wrote a query that looks like this:
SELECT date, group, value FROM [table] GROUP BY date, group, value
But how do I transpose this so that each colums is a date and each row is a collection of totals for the value?
There is no nice way of doing this in BigQuery as of yet, but you can do it following below idea
Step 1
Run below query
SELECT 'SELECT [group], ' +
GROUP_CONCAT_UNQUOTED(
'SUM(IF([date] = "' + [date] + '", value, NULL)) as [d_' + REPLACE([date], '/', '_') + ']'
)
+ ' FROM YourTable GROUP BY [group] ORDER BY [group]'
FROM (
SELECT [date] FROM YourTable GROUP BY [date] ORDER BY [date]
)
As a result - you will get string like below (it is formatted below for readability sake)
SELECT
[group],
SUM(IF([date] = "date1", value, NULL)) AS [d_date1],
SUM(IF([date] = "date2", value, NULL)) AS [d_date2]
FROM YourTable
GROUP BY [group]
ORDER BY [group]
Step 2
Just run above composed query
Result will be like below
group d_date1 d_date2
group1 15 30
Note 1: Step 1 is helpful if you have many groups to pivot so too much of manual work. In this case - Step 1 helps you to generate your query
Note 2: these steps are easily implemented in any client of your choice or you can just run those in BigQuery Web UI
You can see more about pivoting in my other posts.
How to scale Pivoting in BigQuery?
Please note – there is a limitation of 10K columns per table - so you are limited with 10K organizations.
You can also see below as simplified examples (if above one is too complex/verbose):
How to transpose rows to columns with large amount of the data in BigQuery/SQL?
How to create dummy variable columns for thousands of categories in Google BigQuery?
Pivot Repeated fields in BigQuery
Related
I have to get result from column which name is generated by the data from another column. I will explain with the following example:
column names in the database:
months | am1 | am2 | am3 | am4 | am5 | am6 | am7 | am8 | am9 | am10 |am11 | am12
I want in my query to get the value from column starting with am + months value
my current query is:
$query = 'select id, iid, contractnumber, concat("am",`months`) as amount from credits where iid > 0';
but this instead of returning the value (71) of the specific am.. column it returns the column name, for example am5
How can I directly access the value of column am5
Thank you for your time !
That is a bad design and you should change it, but if that's what you got, you can use CASE
select id, iid, contractnumber,
case months
when 1 then am1
when 2 then am2
when 3 then am3
when 4 then am4
when 5 then am5
when 6 then am6
when 7 then am7
when 8 then am8
when 9 then am9
when 10 then am10
when 11 then am11
when 12 then am12
end as amount
from credits
where iid > 0
I have a table users but i have shown only 2 columns I want to sum all the characters of name column.
+----+-------+
| id | name |
+----+-------+
| 0 | user |
| 1 | admin |
| 3 | edit |
+----+-------+
for example ascii sum of user will be
sum(user)=117+115+101+114=447
i have tired this
SELECT ASCII(Substr(name, 1,1)) + ASCII(Substr(name, 2, 1)) FROM user
but it only sums 2.
You are going to have to fetch one character at a time to do the sum. One method is to write a function with a while loop. You can do this with a SELECT, if you know the longest string:
SELECT name, SUM(ASCII(SUBSTR(name, n, 1)))
FROM user u JOIN
(SELECT 1 as n UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL
SELECT 4 UNION ALL SELECT 5 -- sufficient for your examples
) n
ON LENGTH(name) <= n.n
GROUP BY name;
If your goal is to turn the string as something that can be easily compared or a fixed length, then you might consider the encryption functions in MySQL. Adding up the ASCII values is not a particularly good hash function (because strings with the same characters in different orders produce the same value). At the very least, multiplying each ASCII value by the position is a bit better.
My table is like this:
create table alphabet_soup(
id numeric,
index json bigint
);
my data looks like this:
(id, json) looks like this: (1, '{('key':1,'value':"A"),('key':2,'value':"C"),('key':3,'value':"C")...(600,"B")}')
How do I sum across the json for number of A and number of B and do % of the occurence of A or B? I have about 6 different types of values (ABCDEF), but for simplicity I am just looking for a comparison of 3 values.
I am trying to find something to help me calculate the % of occurrence of a value from a key value pair in json. I am using postgres 9.4. I am new to both json and postgres, and I am landing on the same json functions manual page of postgres over and over.
I have managed to find a sum, but how to calculate the % in a nested select and display the key and values in increasing order of occurence like follows:
value | occurence | %
====================================
A | 300 | 50
B | 198 | 33
C | 102 | 17
The script I am using for the sum is :
select id, index->'key'::key as key
sum(case when (1,index::json->'1')::text = (1,index::json->'2')::text
then 1
else 0
end)/count(id) as res
from
alphabet_soup
group by id;
limit 10;
I get an output as follows:
column "alphabet_soup.id" must appear in the group by clause or be used in an aggregate function.
Thanks for the comment Patrick. Sorry I forgot to add I am using postgres 9.4
The easiest way to do this is to expand the json document into a regular row set using the json_each_text() function. Every single json document then becomes a set of rows and you can then apply aggregate function as you would on any other row set. However, you need to use the function as a row source (section 7.2.1.4) (since it returns a set of rows) and then select the value field which has the category of interest. Note that the function uses a field of the table, through an implicit LATERAL join (section 7.2.1.5).
SELECT id, value
FROM alphabet_soup, json_each_text("index");
which yields something like:
test=# SELECT id, value FROM alphabet_soup, json_each_text("index");
id | value
----+-------
1 | A
1 | C
1 | C
1 | B
To this you can apply regular aggregate functions over the appropriate windows to get the result you are looking for:
SELECT DISTINCT id, value,
count(value) OVER (PARTITION BY id, value) AS occurrence,
count(value) OVER (PARTITION BY id, value) * 100.0 /
count(id) OVER (PARTITION BY id) AS percentage
FROM (
SELECT id, value
FROM alphabet_soup, json_each_text("index") ) sub
ORDER BY id, value;
Which gives a result like:
id | value | occurrence | percentage
----+-------+------------+---------------------
1 | A | 1 | 25.0000000000000000
1 | B | 1 | 25.0000000000000000
1 | C | 2 | 50.0000000000000000
This will work for any number of categories (ABCDEF) and any number of ids.
# Patrick, it was an accident. I am new to stackoverflow. I did not realize how ti works. I was fiddling around and I found the answer to the question I asked in addition to the first one. Sorry about that!
For fun, I added some more to the code to make the % compare of the result set:
With q1 as
(SELECT DISTINCT id, value,
count(value) OVER (PARTITION BY id, value) AS occurrence,
count(value) OVER (PARTITION BY id, value) * 100.0 / count(id) OVER(PARTITION BY id) AS percentage
FROM ( SELECT id, value FROM alphabet_soup, json_each_text("index") ) sub
ORDER BY id, value) Select distinct id, value, least(percentage) from q1
Where (least(percentage))>20 Order by id, value;
The output for this is:
id | value | least
----+-------+--------
1 | B | 33
1 | C | 50
I has been working in a updation of a existing website. In that there was a entry form which will save in table... table structure and sample data as follows
id | name | type | in_date | year
-----------------------------------------------------
1 | name1 | 1 | 2-July | 2011
2 | name2 | 2 | 2-June | 2011
3 | name44 | 2 | 8-Sep | 2011
Now I need to order this table in whole date wise ie ( as 2-June-2011) as a simple query
SELECT * FROM order_list order by date DESC
Is any way to do this action ? I tried a lot of query .... Any way to combine these 2 rows ..
We cant alter the DB since it contains more existing records ..
You should store your dates as MySQL DATE types, rather than as strings:
ALTER TABLE order_list ADD COLUMN new_date DATE;
UPDATE order_list
SET new_date = STR_TO_DATE(CONCAT(in_date, '-', year), '%e-%b-%Y');
ALTER TABLE order_list DROP COLUMN in_date, DROP COLUMN year;
Ordering then becomes trivial (i.e. will work exactly as you have attempted):
SELECT * FROM order_list ORDER BY date DESC;
If you're unable to alter the database schema, you can perform the STR_TO_DATE operation in the ORDER BY clause (but this is not very efficient):
SELECT *
FROM order_list
ORDER BY STR_TO_DATE(CONCAT(in_date, '-', year), '%e-%b-%Y') DESC
Don't do that. Put the entire date in one column, and then, if you really have to, create computed columns that will hold the year or day/month.
You can create a simple script that will integrate those two existing columns into the one united-date column in your existing database.
You can try with + (SQL SERVER) or CONCAT (MySQL)
SELECT * FROM order_list order by in_date + year DESC
Thank you for an example. I'm a trouble to figure out a statement with interval...
I need calculate users in certain interval of dates, but interval must be entered by a user. Example: from '2010/05/05' to '2010/30/07' if interval is 1m(month), then total of this users each 1m interval, like this: 2010/05/05 to 2010/06/05 is total users.
So far I got:
SELECT col1, client, COUNT(client) FROM table1, table2 WHERE col1 IN (condition) AND date BETWEEN '2010/05/01' AND '2010/07/30' AND DATE_ADD(CURDATE(),INTERVAL + 1 month) GROUP BY client;
Of course it calculates all total, but not dates with interval.
Also I tried to use Perl
my #data; #data from dbase.
%date_hash = ($data[1] =>$total); #$data[1] is beg and end dates user entered
foreach $dates (values %date_hash) {
$date_hash{$dates}=$total;
print "Print hash: $dates $date_hash{$dates} \n"
Thank you in advance, :)
One possible SQL only solution (I will offer an algorithm, not exact SQL code)
Create a temp table INTERVALS, populate it (probably easiest to do from Perl in a loop, though MYSQL loop would be sufficient) with data as follows (from 5/5/2010 to 12/10/2010, 1 month intervals):
| start_period | end_period | interval_number |
===============================================
| 2010/05/05 | 2010/06/04 | 1 |
| 2010/06/05 | 2010/07/04 | 2 |
| ...
| 2010/12/05 | 2010/12/10 | 8 |
Then, run the query joining your table to INTERVALS temp table via
SELECT client, COUNT(client)
, i.interval_number, i.start_period, i.end_period
FROM table1
WHERE col1 IN (condition)
AND table1.date >= inetrvals.start_period
AND table1.date <= inetrvals.end_period
GROUP BY client, i.interval_number, i.start_period, i.end_period
Please note that you can select other columns from table1 but only if you group on them as well.