Select mysql data in un-normalised form - mysql

Question

Apologies if this seems a 'stupid' question - I don't really know the right term to describe what I am trying to do (and thus searching for help on it was bit fruitless).
Basically, I initially had data that was in the form:
| timestamp | category A | category B | .......| category n|
| 2011-12-02 00:05:00 | 23.63 | 27.00 | .......| 24.03 |
| 2011-12-02 00:10:00 | 23.75 | 24.42 | .......| 24.45 |
| 2011-12-02 00:15:00 | 23.31 | 23.96 | .......| 26.54 |
I put this data into a database (and normalised it) so that it exists in the database as follows:
+---------------------+--------------+-------+
| timestamp | catergory_id | value |
+---------------------+--------------+-------+
| 2011-12-02 00:05:00 | 2 | 27.00 |
| 2011-12-02 00:10:00 | 2 | 24.42 |
| 2011-12-02 00:15:00 | 2 | 23.96 |
| 2011-12-02 00:20:00 | 2 | 23.73 |
| 2011-12-02 00:25:00 | 2 | 23.73 |
+---------------------+--------------+-------+
What I am trying to select different categories by timestamp (to enable comparison) , like so:
+---------------------+-------+-------+
| timestamp | cat_a | cat_b |
+---------------------+-------+-------+
| 2011-12-02 00:05:00 | 23.63 | 27.00 |
| 2011-12-02 00:10:00 | 23.75 | 24.42 |
| 2011-12-02 00:15:00 | 23.31 | 23.96 |
| 2011-12-02 00:20:00 | 23.00 | 23.73 |
| 2011-12-02 00:25:00 | 22.91 | 23.73 |
+---------------------+-------+-------+
This is basically similar to the original data structure (But I would like to select/compare between multiple and variable, categories not just two).
I have been able to to this using join (after selecting the individual categories in individual tables). This is okay for say comparing across two categories, but seems quite inefficient, particularly if I want to select say 15 or 20 different categories to compare. It is also problematic if a particular category is missing a data point.
(The other way I have been doing this is by selecting individual tables and later "merging" the data in the python application in which it is later used, but this also seems equally inefficient)
I feel like there must be an easier or more intuitive way to do this in mysql- and I am just missing something quite basic. I don't really want to de-normalize (As there is a lot of categories, and it makes sense to have it normalized for other uses, besides this one).
Cheers,

Answer 1

This is basically a pivot table problem. MySQL doesn't have a built-in SQL extension to make pivot tables like some other DBMSs do so they are a bit tricky. You can find one way of making them here: http://www.artfulsoftware.com/infotree/qrytip.php?id=78

Answer 2

My solution to this problem used the python data tool pandas. (This won't suit those interested in a pure MySQL solution - for this case, check out Joni's solutions above, or have a look at some of the similar stackoverflow answers e.g. mysql pivot query results with GROUP BY or MySQL pivot table query with dynamic columns).
Firstly I created a pandas dataframe with the data I wanted to select/compare (using the sql.read_frame method from pandas.io, and the appropriate sql_query):
df=sql.read_frame(sql_query,DB_connection)
This created a dataframe as such:
df.head():
timestamp category_id value
0 2011-01-01 00:00:00 4 22.05
1 2011-01-01 00:05:00 4 24.10
2 2011-01-01 00:10:00 4 23.98
3 2011-01-01 00:15:00 4 24.10
4 2011-01-01 00:20:00 4 24.10
This was then "pivoted" using the pandas.pivot_table method:
df2=df.pivot_table(rows='timestamp',cols='category_id',values='value')
Which create the exact output I was after:
df2.head():
category_id 2 4 5 6 7
timestamp
2011-01-01 00:00:00 23.43 22.05 25.07 19.47 21.32
2011-01-01 00:05:00 25.31 24.10 25.69 21.32 22.94
2011-01-01 00:10:00 25.31 23.98 24.84 21.32 22.59
2011-01-01 00:15:00 25.31 24.10 25.47 21.10 21.39
2011-01-01 00:20:00 25.31 24.10 25.69 20.01 17.9
Hope someone else finds this useful!