I've been pulling my hair out for two days trying to put a MySQL query together to pull some data together. This is beyond my level of experience with SQL, however.
The following query pulls the data that I need, but I need to format it differently so I can use it.
SELECT b.key, a.author, a.comment FROM action a, issue b
WHERE a.issueid=b.id AND a.actiontype='comment';
This produces output like so:
1 | joe | comment 3
3 | sally | comment 2
1 | sam | comment 2
2 | julie | comment 2
1 | bill | comment 1
3 | joe | comment 1
2 | sam | comment 1
b.key is unique and can contain 0-∞ a.comment. There is a 1-1 relationship with the a.author and a.comment and I must maintain the original association. The output seems to be in ascending order based on the date of a.comment.
I would like to do get the data into the following format (CSV,) merging the a.author and a.comment results into a single column:
"1", "bill - comment 1", "sam - comment 2", "joe - comment 1"
"2", "julie - comment 1", "sam - comment 2"
"3", "joe - comment 1", "sally - comment 2"
I played around with several combinations of GROUP_CONCAT and ORDER and also messed with subqueries a bit based on this post and can't seem to get it formatted the way I need. Additionally, it seems like the order of the comment column changes. I need to maintain the original (oldest to newest) order, but my original query above produces results in newest to oldest order. If the delimiters pose a problem, I can certainly do some post processing with perl or awk search & replace on the results. Any help is greatly appreciated!
How about
SELECT
issue.key,
GROUP_CONCAT(CONCAT(action.author, ' - ', action.comment) ORDER BY action.created ASC SEPARATOR '|')
FROM action
INNER JOIN issue
ON action.issueid = issue.id
WHERE action.actiontype = 'comment'
GROUP BY issue.key;
with some post processing to split based on the pipe character. If you get truncation issues you may need to increase the value of group_concat_max_len.
Check this out. The join doesn't affect in any way how data is displayed so I'm writing the query as if your table is actually the one you display as the output
select concat('"', id, '", ',
group_concat(concat('"', author, ' - ', comment, '"') separator ", ")
) as result from table1
group by id
The ordering of the group_concat is up to you.
Here is an example.
Hope this helps.
Related
I'm working with a table that uses a JSON column to store questions and answers as an array of objects where the key is the question and the value is the answer.
The questions are user generated so the keys can literally be anything.
For example:
id
questions
1
[{"Favourite Food?": "Pizza"}, {"Age?": "12"}]
2
[{"Where do you live?": "France"}, {"Are you ok?", "No, not really"}]
I'm trying to figure out if it's possible using MySQL to query this data and get a result that looks like this:
question
answer
"Favourite Food?"
"Pizza"
"Age?"
"12"
"Where do you live?"
"France"
"Are you ok?"
"No, not really"
I didn't think this would be too hard but quickly realized this is probably way out of my depth with MySQL!
The main struggle here is that I can't seem to figure out how to extract the keys when they're buried in an Array like this and I don't know what the names of the keys would be.
The closest I can get is a query that looks like this:
select j.* from ats_applicants,
JSON_TABLE(additional_fields, "$[*].*" COLUMNS ( question varchar(100) PATH '$', answer varchar(100) PATH '$'))
as j where additional_fields is not null;
Problem is this only gives me the values of the objects and not the keys like this:
question
answer
"Pizza"
"Pizza"
"12"
"12"
"etc"
"etc"
I have no good reason for doing this by the way. Purely scratching an itch because I saw this problem come up and thought it would be simple -- and maybe it is -- I just don't see how to do this!
Thanks in advance 🙏
It would be easier if you store the JSON differently:
id
questions
1
[{"question": "Favourite Food?", "answer": "Pizza"}, {"question": "Age?": "answer": "12"}]
2
[{"question": "Where do you live?", "answer": "France"}, {"question": "Are you ok?", "answer": "No, not really"}]
Then you can use JSON_TABLE() to map the fields into columns:
SELECT j.*
FROM ats_applicants
CROSS JOIN JSON_TABLE(additional_fields, '$[*]' COLUMNS (
question varchar(100) PATH '$.question',
answer varchar(100) PATH '$.answer'
)
) AS j
WHERE additional_fields IS NOT NULL;
But it would be even easier if you don't use JSON at all, but just store each question and answer in a row by itself:
id
applicant_id
question
answer
1
1
Favorite Food?
Pizza
2
1
Age?
12
3
2
Where do you live?
France
4
2
Are you ok?
No, not really
People are too eager to use JSON these days. Normal rows and columns are easier and more efficient.
Re your comment:
Yes, I believe it's possible, but it's more trouble than it's worth.
You can't use JSON_TABLE() directly, because the objects don't have consistent key names.
So you would need to:
Split the array into one object per row by joining to a set of integers. For example, use a temp table or a CTE.
For each object, use JSON_KEYS() to get the key names into an array.
Use those key names in calls to JSON_EXTRACT().
Below is a solution I got to work. To my mind, this is utterly stupid to use a data model that requires such complex code to do the simplest task.
WITH RECURSIVE n AS (
SELECT 0 AS n
UNION SELECT n+1 FROM n WHERE n < 1
),
pair AS (
SELECT id, n.n, JSON_EXTRACT(additional_fields, CONCAT('$[', n.n, ']')) AS pair
FROM n CROSS JOIN ats_applicants
),
q AS (
SELECT id, n, JSON_UNQUOTE(JSON_EXTRACT(JSON_KEYS(pair), '$[0]')) AS question
FROM pair
)
SELECT id, q.question, JSON_UNQUOTE(JSON_EXTRACT(pair.pair, CONCAT('$."', q.question, '"'))) AS answer
FROM q JOIN pair USING (id, n)
Output:
+----+--------------------+----------------+
| id | question | answer |
+----+--------------------+----------------+
| 1 | Age? | 12 |
| 1 | Favourite Food? | Pizza |
| 2 | Are you ok? | No, not really |
| 2 | Where do you live? | France |
+----+--------------------+----------------+
I have a table like this
id | user_id | code | type | time
-----------------------------------
2 2 fdsa r 1358300000
3 2 barf r 1358311000
4 2 yack r 1358311220
5 3 surf r 1358311000
6 3 yooo r 1358300000
7 4 poot r 1358311220
I want to get the concatenated 'code' column for user 2 and user 3 for each matching time.
I want to receive a result set like this:
code | time
-------------------------------
fdsayooo 1358300000
barfsurf 1358311000
Please note that there is no yackpoot code because the query was not looking for user 4.
You can use GROUP_CONCAT function. Try this:
SELECT GROUP_CONCAT(code SEPARATOR '') code, time
FROM tbl
WHERE user_id in (2, 3)
GROUP BY time
HAVING COUNT(time) = 2;
SQL FIDDLE DEMO
What you are looking for is GROUP_CONCAT, but you are missing a lot of details in your question to provide a good example. This should get you started:
SELECT GROUP_CONCAT(code), time
FROM myTable
WHERE user_id in (2, 3)
GROUP BY time;
Missing details are:
Is there an order required? Not sure how ordering would be done useing grouping, would need to test if critical
Need other fields? If so you will likely end up needing to do a sub-select or secondary query.
Do you only want results with multiple times?
Do you really want no separator between values in the results column (specify the delimiter with SEPARATOR '' in the GROUP_CONCAT
Notes:
You can add more fields to the GROUP BY if you want to do it by something else (like user_id and time).
I think it will be easiest to start with the table I have and the result I am aiming for.
Name | Date
A | 03/01/2012
A | 03/01/2012
B | 02/01/2012
A | 02/01/2012
B | 02/01/2012
A | 02/01/2012
B | 01/01/2012
B | 01/01/2012
A | 01/01/2012
I want the result of my query to be:
Name | 01/01/2012 | 02/01/2012 | 03/01/2012
A | 1 | 2 | 2
B | 2 | 2 | 0
So basically I want to count the number of rows that have the same date, but for each individual name. So a simple group by of dates won't do because it would merge the names together. And then I want to output a table that shows the counts for each individual date using php.
I've seen answers suggest something like this:
SELECT
NAME,
SUM(CASE WHEN GRADE = 1 THEN 1 ELSE 0 END) AS GRADE1,
SUM(CASE WHEN GRADE = 2 THEN 1 ELSE 0 END) AS GRADE2,
SUM(CASE WHEN GRADE = 3 THEN 1 ELSE 0 END) AS GRADE3
FROM Rodzaj
GROUP BY NAME
so I imagine there would be a way for me to tweak that but I was wondering if there is another way, or is that the most efficient?
I was perhaps thinking if the while loop were to output just one specific name and date each time along with the count, so the first result would be A,01/01/2012,1 then the next A,02/01/2012,2 - A,03/01/2012,3 - B,01/01/2012,2 etc. then perhaps that would be doable through a different technique but not sure if something like that is possible and if it would be efficient.
So I'm basically looking to see if anyone has any ideas that are a bit outside the box for this and how they would compare.
I hope I explained everything well enough and thanks in advance for any help.
You have to include two columns in your GROUP BY:
SELECT name, COUNT(*) AS count
FROM your_table
GROUP BY name, date
This will get the counts of each name -> date combination in row-format. Since you also wanted to include a 0 count if the name didn't have any rows on a certain date, you can use:
SELECT a.name,
b.date,
COUNT(c.name) AS date_count
FROM (SELECT DISTINCT name FROM your_table) a
CROSS JOIN (SELECT DISTINCT date FROM your_table) b
LEFT JOIN your_table c ON a.name = c.name AND
b.date = c.date
GROUP BY a.name,
b.date
SQLFiddle Demo
You're asking for a "pivot". Basically, it is what it is. The real problem with a pivot is that the column names must adapt to the data, which is impossible to do with SQL alone.
Here's how you do it:
SELECT
Name,
SUM(`Date` = '01/01/2012') AS `01/01/2012`,
SUM(`Date` = '02/01/2012') AS `02/01/2012`,
SUM(`Date` = '03/01/2012') AS `03/01/2012`
FROM mytable
GROUP BY Name
Note the cool way you can SUM() a condition in mysql, becasue in mysql true is 1 and false is 0, so summing a condition is equivalent to counting the number of times it's true.
It is not more efficient to use an inner group by first.
Just in case anyone is interested in what was the best method:
Zane's second suggestion was the slowest, I loaded in a third of the data I did for the other two and it took quite a while. Perhaps on smaller tables it would be more efficient, and although I am not working with a huge table roughly 28,000 rows was enough to create significant lag, with the between clause dropping the result to about 4000 rows.
Bohemian's answer gave me the least amount to code, I threw in a loop to create all the case statements and it worked with relative ease. The benefit of this method was the simplicity, besides creating the loop for the cases, the results come in without the need for any php tricks, just simple foreach to get all the columns. Recommended for those not confident with php.
However, I found Zane's first suggestion the quickest performing and despite the need for extra php coding it seems I will be sticking with this method. The disadvantage of this method is that it only gives the dates that actually have data, so creating a table with all the dates becomes a bit more complicated. What I did was create a variable that keeps track of what date it is supposed to be compared to the table column which is reset on each table row, when the result of the query is equal to that date it echoes the value otherwise it does a while loop echoing table cells with 0 until the dates do match. It also had to do a check to see if the 'Name' value is still the same and if not it would switch to the next row after filling in any missing cells with 0 to the end of that row. If anyone is interested in seeing the code you can message me.
Results of the two methods over 3 months of data (a column for each day so roughly 90 case statements) ~ 12,000 rows out of 28,000:Bohemian's Pivot - ~0.158s (highest seen ~0.36s)Zane's Double Group by - ~0.086s (highest seen ~0.15s)
Firstly I'd like to start by apologizing for the potentially miss-leading title... I am finding it difficult to describe what I am trying to do here.
With the current project I'm working on, we have setup a 'dynamic' database structure with MySQL that looks something like this.
item_details ( Describes the item_data )
fieldID | fieldValue | fieldCaption
1 | addr1 | Address Line 1
2 | country | Country
item_data
itemID | fieldID | fieldValue
12345 | 1 | Some Random Address
12345 | 2 | United Kingdom
So as you can see, if for example I wanted to lookup the address for the item 12345 I would simply do the statement.
SELECT fieldValue FROM item_data WHERE fieldID=1 and itemID=12345;
But here is where I am stuck... the database is relatively large with around ~80k rows and I am trying to create a set of search functions within PHP.
I would like to be able to perform a query on the result set of a query as quickly as possible...
For example, Search an address name within a certain country... ie: Search for the fieldValue of the results with the same itemID's as the results from the query:
'SELECT itemID from item_data WHERE fieldID=2 and fieldValue='United Kingdom'..
Sorry If I am unclear, I have been struggling with this for the past couple of days...
Cheers
You can do this in a couple of ways. One is to use multiple joins to the item_data table with the fieldID limited to whatever it is you want to get.
SELECT *
FROM
Item i
INNER JOIN item_data country
ON i.itemID = country.itemID
and fieldid = 2
INNER JOIN item_data address
ON i.itemID = country.itemID
and fieldid = 1
WHERE
country.fieldValue= 'United Kingdom'
and address.fieldValue= 'Whatever'
As an aside this structure is often referred to as an Entry Attribute Value or EAV database
Sorry in advance if this sounds patronizing, but (as you suggested) I'm not quite clear what you are asking for.
If you are looking for one query to do the whole thing, you could simply nest them. For your example, pretend there is a table named CACHED with the results of your UK query, and write the query you want against that, but replace CACHED with your UK query.
If the idea is that you have ALREADY done this UK query and want to (re-)use its results, you could save the results to a table in the DB (which may not be practical if there are a large number of queries executed), or save the list of IDs as text and paste that into the subsequent query (...WHERE ID in (...) ... ), which might be OK if your 'cached' query gives you a manageable fraction of the original table.
I've racked my brains and googled extensively to find a solution and I suspect I may not be asking the question clearly so please bear with me.
I've got to build a couple of queries that filter records on the following basis. Although multiple tables involved in extracting the data I'll stick the basic requirement.
The following are the sample values:
Key | Decision
123 | Complete
123 | Additional info
123 | Something
123 | Complete
.
.
.
254 | Complete
254 | Complete
254 | Complete
.
.
.
Based on the above data I can do a select and group by Key and Decision to get data set as follows:
Key | Decision
123 | Complete
123 | Additional info
123 | Something
.
.
.
254 | Complete
.
.
.
The actual data I need is of two types (these are separe queries that have to be built)
1) Keys where the only decision is "Complete" - In the above example only Key=254 would match
2) Keys where decision could contain "Additional info" - In the above example only Key=123 would match
It seems almost possible, like I have the answer floating around somewhere, and I can't quite grasp it. Or is this wishful thinking?
I did try the following
select key from table where decision not in (select key from table where decision <> "Complete")
This gets me the result I want for Decision=Complete. However, with the final selection being at least containing at least three joins, I suspect that the performance is going to be bad. The queries will be executed on Oracle 11g.
If anyone has suggestions that helps me get out of this ideas rut, I would deeply appreciate it.
For the first question
select `key` from your_table
group by key
having count(decision) = sum(decision="complete")
for the second one
select `key` from your_table
where decision = 'Additional Info'
group by `key`
1) Keys where the only decision is "Complete" - In the above example only Key=254 would match
select key
from table
group
by key
having min(decision) = 'Complete'
and max(decision) = 'Complete'
or what #nick rulez wrote with the following modification (to make it run on Oracle as well):
having count(decision) = sum(case when decision = 'Complete' then 1 else 0 end)
2) Keys where decision could contain "Additional info" - In the above example only Key=123 would match
select distinct key
from table
where decision = 'Additional info';