I have a mysql table which is filled with inputs from a webform on my website. The form has fields for last name, surname, email, phone, address, etc.... and when a user submits the form these data are stored in a mysql table in a rather strange way.
my table looks like this:
subission# | value | field | tstamp | and |many |more |columns
=====================================================================================
1 |john#server.com |email |1448898875 | | | |
1 |john |firstname|1448898875 | | | |
1 |doe |lastname |1448898875 | | | |
1 |london |city |1448898875 | | | |
2 |jane#aol.com |email |1448898870 | | | |
2 |jane |firstname|1448898870 | | | |
2 |doe |lastname |1448898870 | | | |
2 |new york |city |1448898870 | | | |
3 |tim #aol.com |email |1448838571 | | | |
3 |tim |firstname|1448838571 | | | |
3 |smith |lastname |1448838571 | | | |
3 |paris |city |1448838571 | | | |
I need to export these data to a csv file in order to import it to a newsletter script on some other server, but the server expects these data in a different format:
submission#,email,firstname,lastname,tstamp,.....
1,john#server.com,john,doe,london,1448898875,,,,
2,jane#aol.com,jane,doe,1448898870,,,,
The export as csv is not the problem, but how do I get all the data of one submission# into one row? Can anyone please point me into the right direction, how to accomplish this with SQL?
You can achieve the desired output, if you concatenate the field contents into a single field using concat() and group_concat() functions, where the values are separated by comma.
The only issue can be if for a particular submission any of the properies is missing. If that's the case, then you will need a helper table which lists all properies and you need to left join on that table. Since this is not the case for your sample data, I'm not providing the code for this scenario.
select concat(submission, ',', group_concat(`value` order by `field` asc), ',',tstamp)
from table group by submission, tstamp
If you need the field names in the 1st row, then create a separate query that conatenates the field names separated by commas and combine the 2 with union.
Related
I have table and I need to transform it at horizontal mode
| id | Number | Name |
|------|------------|--------|
| 3695 | 0445458225 | Name1 |
| 3695 | 0445458228 | Name2 |
| 3695 | 0445458553 | Name3 |
| 3695 | 0445458560 | Name4 |
| 3695 | 0445458551 | Name5 |
| 3695 | 0445458561 | Name7 |
| 3695 | 0445458522 | Name8 |
| 3695 | 0445458543 | Name9 |
| 3696 | 0445458226 | Name10 |
| 3696 | 0445458540 | Name11 |
| 3696 | 0445458543 | Name12 |
| 3696 | 0445923962 | Name13 |
| 3696 | 0500266382 | Name14 |
| 3697 | 0445923962 | Name15 |
| 3697 | 0445458226 | Name16 |
| 3697 | 0500266382 | Name17 |
| 3697 | 0445458564 | Name18 |
I got it as below:
| id | Name1 | Name2 | Name3 | Name4 | Name5 | Name6 | Name7 | Name8 | Name9 | Name10 | Name11 | Name12 | Name13 |
|------ |-----------|---------- |-------|----------|----------- |-------|-------|-------|----------|--------|---------- |----------|----------|
| 3695 | 445458553 | 44518551 | | 44548560 | | | | | 44548228 | | 44548543 | | 44548225 |
| 3696 | | | | | 445923962 | | | | | | 44548543 | 44548226 | |
| 3694 | | | | | 445923962 | | | | | | 44548543 | 44548226 | |
But what I need is this:
| id | Member1 | Member2 | Member3 | Member4 | Member5 | Member6 | Member7 | Member8 | Member9 | Member10 | member11 | Name12 | Name13 |
|------|-----------|----------|----------|----------|----------|----------|---------|---------|---------|----------|----------|--------|--------|
| 3695 | 445458553 | 44518551 | 44548560 | 44548228 | 44548543 | 44548225 | | | | | | | |
| 3696 | 445923962 | 44548543 | 44548226 | | | | | | | | | | |
| 3694 | 445923962 | 44548543 | 44548226 | | | | | | | | | | |
So no empty cells, column should be named as last table also original id, numbers and names are different
every time when query runs (different group id's)
Here is my SQL-code:
TRANSFORM Avg(Nick_agents.Number) AS FirstOfNumber
SELECT Nick_agents.id
FROM Nick_agents
GROUP BY Nick_agents.id
PIVOT Nick_agents.Name;
Access has a built-in Crosstab Query Wizard that does this for you.
On the Create tab, in the Queries group, click Query Wizard.
The Queries group in the Access ribbon displays two options: Query Wizard and Query Design
In the New Query dialog box, click Crosstab Query Wizard, and then click OK.
The Crosstab Query Wizard starts.
On the first page of the wizard, choose the table or query that you want to use to create a crosstab query. For this example, select the Products table and then click Next.
[![enter image description here][2]][2]
On the next page, choose the field that contains the values that you want to use as row headings. You can select up to three fields to use as row headings sources, but the fewer row headings you use, the easier your crosstab datasheet will be to read. If you choose more than one field to supply row headings, the order in which you choose the fields determines the default order in which your results are sorted.
For this example, select Supplier IDs.Value and then click the button labeled with a > symbol. Notice that Access displays the field name along the left side of the sample query preview at the bottom of the dialog box. Click Next to continue.
Select a field to display as row headings on the Crosstab query wizard.
On the next page, choose the field that contains the values that you want to use as column headings. In general, you should choose a field that contains few values, to help keep your results easy to read. For example, using a field that has only a few possible values (such as gender) might be preferable to using a field that can contain many different values (such as age).
If the field that you choose to use for column headings has the Date/Time data type, the wizard adds a step that lets you specify how to group the dates into intervals, such as months or quarters.
For this example, select Category and notice that Access displays category sample names along the top of the sample query preview at the bottom of the dialog box. Click Next to continue.
Select a field to display as row headings on the Crosstab query wizard.
If you choose a Date/Time field for column headings, the next page of the wizard asks you to specify the interval to use to group the dates. You can specify Year, Quarter, Month, Date, or Date/Time. If you do not choose a Date/Time field for column headings, the wizard skips this page.
On the next page, choose a field and a function to use to calculate summary values. The data type of the field that you select determines which functions are available.
On the same page, select or clear the Yes, include row sums check box to include or exclude row sums.
If you include row sums, the crosstab query has an additional row heading that uses the same field and function as the field value. Including a row sum inserts an additional column that summarizes the remaining columns. For example, if your crosstab query calculates average age by location and gender (with gender column headings), the additional column calculates the average age by location, across all genders.
For this example, select ID in the Fields box and Count in the Functions box in order to have Access count the number of products in each intersection of supplier and category. Leave the Yes, include row sums check box selected. Access will create a column that totals the number of products from each supplier. Click Next to continue.
Select a field and function to calculate on the Crosstab query wizard.
On the last page of the wizard, type a name for your query and then specify whether you want to view the results or modify the query design.
You can change the function that is used to produce row sums by editing the crosstab query in Design view.
If you've walked through this example using the Products table from the Northwind database, the crosstab query displays the list of supplier names as rows, the product category names as columns, and a count of the number of products in each intersection.
In your example, your data is completely different. In the first example, you have Name1, Name2, Name3, as field names and in the second example, you have Member1, Member2, Member3 as field names. Let's say this is your actual data.
id Number Name
3697 445923962 Member1
3696 445923962 Member1
3695 445458553 Member1
3696 445458543 Member2
3697 445458543 Member2
3695 445458543 Member2
3695 445458551 Member2
3695 445458560 Member3
3696 445458226 Member3
3697 445458226 Member3
Then, this is your SQL.
TRANSFORM First(Table1.[Number]) AS FirstOfNumber
SELECT Table1.[id], First(Table1.[Number]) AS [Total Of Number]
FROM Table1
GROUP BY Table1.[id]
PIVOT Table1.[Name];
This is the Design View.
When you run the SQL, you get this.
Make sense???
I have two tables in a MySQL database like this:
User:
userid |userid | Username | Plan(VARCHAR) | Status |
-----------+------------+--------------+---------------+---------+
1 | 1 | John | 1,2,3 |1 |
2 | 2 | Cynthia | 1,2 |1 |
3 | 3 | Charles | 2,3,4 |1 |
Plan: (planid is primary key)
planid(INT) | Plan_Name | Cost | status |
-------------+----------------+----------+--------------+
1 | Tamil Pack | 100 | ACTIVE |
2 | English Pack | 100 | ACTIVE |
3 | SportsPack | 100 | ACTIVE |
4 | KidsPack | 100 | ACTIVE |
OUTPUT
id |userid | Username | Plan | Planname |
---+-------+----------+------------+-------------------------------------+
1 | 1 | John | 1,2,3 |Tamil Pack,English Pack,SportsPack |
2 | 2 | Cynthia | 1,2 |Tamil Pack,English Pack |
3 | 3 | Charles | 2,3,4 |English Pack,Sportspack, Kidspack |
Since plan id in Plan table is integer and the user can hold many plans, its stored as comma separated as varchar, so when i try with IN condition its not working.
SELECT * FROM plan WHERE find_in_set(plan_id,(select user.planid from user where user.userid=1))
This get me the 3 rows from plan table but i want the desired output as above.
How to do that.? any help Please
A rewrite off your query what should work is as follows..
Query
SELECT
all columns you need
, GROUP_CONCAT(Plan.Plan_Name ORDER BY Plan.planid) AS Planname
FROM
Plan
WHERE
FIND_IN_SET(Plan.plan_id,(
SELECT
User.Plan
FROM
user
WHERE User.userid = 1
)
)
GROUP BY
all columns what are in the select (NOT the GROUP_CONCAT function)
You also can use FIND_IN_SET on the ON clause off a INNER JOIN.
One problem is that the join won't ever use indexes.
Query
SELECT
all columns you need
, GROUP_CONCAT(Plan.Plan_Name ORDER BY Plan.planid) AS Planname
FROM
User
INNER JOIN
Plan
ON
FIND_IN_SET(Plan.id, User.Plan)
WHERE
User.id = 1
GROUP BY
all columns what are in the select (NOT the GROUP_CONCAT function)
Like i said in the comments you should normalize the table structures and add the table User_Plan whats holds the relations between the table User and Plan.
In one of our columns we store this example json string:
[{"Name":"Pay Amount","Value":"0.00"},{"Name":"Period","Value":"3"},{"Name":"Client","Value":"TestClient"},{"Name":"Our Reference","Value":""},{"Name":"Pay Type","Value":"Test"}]
We repeat the Names through out and the values will differ.
I've tried querying this data using JsonGet_string :
SELECT
JSONGET_string(Header, "Name") Name
FROM tbl
but what it does it selects the first one i.e PayAmount and it only displays a list of payamount it doesn't select anything for Period, Client etc.
The result that it returns looks like this:
| Name |
|----------|
| |
| PayAmount|
| PayAmount|
And it should return this:
| Name |
|-------------|
| |
| PayAmount |
| Period |
| Client |
| OutReference|
| Pay Type |
Any ideas?
I have a lot of spreadsheets that pull transactional information from our ERP software into Excel using the Microsoft Query that we then perform other calculations on automatically. Recently we upgraded our ERP system, but management made the decision to leave the transactional history in the old databases to have a clean one going forward in the new system. I still need to have some "rolling 12 months" graphs, but if I use only the old database, I'm missing new data and if I use only the new, I'm missing the last 11 months data.
Is there a way that I can write a query in Excel to pull data from the old database PartTran table and merge it with the new database PartTran table without user intervention each time? For instance, I don't want my users (if possible) to have to have two queries that they copy and paste into one Excel table. The schema of the tables (at least the columns I need) are identically named and defined.
If you want to take a bit of a fun, hacky Excel approach, you could do the "copy-paste" bit FOR your users behind the scenes. Given two similar tables OLD and NEW with structures
+-----+------+-------+------------+
| id | foo | bar | date |
+-----+------+-------+------------+
| 95 | blah | $25 | 2015-06-01 |
| 96 | bork | $12 | 2015-07-01 |
| 97 | bump | $200 | 2015-08-01 |
| 98 | fizz | | 2015-09-01 |
| 99 | buzz | $50 | 2015-10-01 |
| 100 | char | ($1) | 2015-11-01 |
| 101 | mope | | 2015-12-01 |
+-----+------+-------+------------+
and
+----+-----+-------+------------+------+---------+
| id | foo | bar | date | fizz | buzz |
+----+-----+-------+------------+------+---------+
| 1 | cat | ($10) | 2016-01-01 | 285B | 1110111 |
| 2 | dog | $25 | 2016-02-01 | 27F5 | 1110100 |
| 3 | ant | $100 | 2016-03-01 | 1F91 | 1001111 |
+----+-----+-------+------------+------+---------+
... you can union together the data for these two datasets with some prudent excel wizardry as below:
Your UNION table ( named using alt+j+t+a ) should have the following items:
New natural ID
DataSet pointer ( name of old or new table )
Derived ID from original dataset
Columns of data you want from Old & New DataSets
example:
+---------+------------+------------+----+------+-----+------------+------+------+
| UnionId | SourceName | SourceRank | id | foo | bar | date | fizz | buzz |
+---------+------------+------------+----+------+-----+------------+------+------+
| 1 | OLD | | | | | | | |
| 2 | NEW | | | | | | | |
+---------+------------+------------+----+------+-----+------------+------+------+
You will then make judicious use of Indirect() and VlookUp() to derive the lookup id and column targets. Sample code below
SourceRank - helper column
=COUNTIFS([SourceName],[#SourceName],[UnionId],"<="&[#UnionId])
id - the id from the original DataSet
=SMALL(INDIRECT([#SourceName]&"[id]"),[#SourceRank])
Everything else is just VlookUp madness!! Although I've taken the liberty of copying the sample code below for reference
foo =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[foo]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
bar =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[bar]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
date =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[date]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
fizz =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[fizz]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
buzz =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[fizz]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
Output
You'll likely want to make prudent use of If() and/or IfError() to help your users ignore the new column references to the old table and those rows that do not yet have data. Without that, however, you'll end up with something like the below.
This is both ready to accept & read new inputs to both OLD and NEW DataSets and is sortable to get rid of those pesky placeholder rows...
Hope this helps! Happy coding!
A previous DBA managed a non relational table with 2.4M entries, all with unique ID's. However, there are duplicate records with different data in each record for example:
+---------+---------+--------------+----------------------+-------------+
| id | Name | Address | Phone | Email | LastVisited |
+---------+---------+--------------+---------+------------+-------------+
| 1 | bob | 12 Some Road | 02456 | | |
| 2 | bobby | | 02456 | bob#domain | |
| 3 | bob | 12 Some Rd | 02456 | | 2010-07-13 |
| 4 | sir bob | | 02456 | | |
| 5 | bob | 12SomeRoad | 02456 | | |
| 6 | mr bob | | 02456 | | |
| 7 | robert | | 02456 | | |
+---------+---------+--------------+---------+------------+-------------+
This isnt the exact table - the real table has 32 columns - this is just to illustrate
I know how to identify the duplicates, in this case i'm using the phone number. I've extracted the duplicates into a seperate table - there's 730k entires in total.
What would be the most efficient way of merging these records (and flagging the un-needed records for deletion)?
I've looked at using UPDATE with INNER JOIN's, but there are several WHERE clauses needed, because i want to update the first record with data from subsequent records, where that subsequent record has additional data the former record does not.
I've looked at third party software such as Fuzzy Dups, but i'd like a pure MySQL option if possible
The end goal then is that i'd be left with something like:
+---------+---------+--------------+----------------------+-------------+
| id | Name | Address | Phone | Email | LastVisited |
+---------+---------+--------------+---------+------------+-------------+
| 1 | bob | 12 Some Road | 02456 | bob#domain | 2010-07-13 |
+---------+---------+--------------+---------+------------+-------------+
Should i be looking at looping in a stored procedure / function or is there some real easy thing i've missed?
U have to create a PROCEDURE, but before that
create ur own temp_table like :
Insert into temp_table(column1, column2,....) values (select column1, column2... from myTable GROUP BY phoneNumber)
U have to create the above mentioned physical table so that u can run a cursor on it.
create PROCEDURE myPROC
{
create a cursor on temp::
fetch the phoneNumber and id of the current row from the temp_table to the local variable(L_id, L_phoneNum).
And here too u need to create a new similar_tempTable which will contain the values as
Insert into similar_tempTable(column1, column2,....) values (Select column1, column2,.... from myTable where phoneNumber=L_phoneNumber)
The next step is to extract the values of each column u want from similar_tempTable and update into the the row of myTable where id=L_id and delete the rest duplicate rows from myTable.
And one more thing, truncate the similar_tempTable after every iteration of the cursor...
Hope this will help u...