I have a report from Cloudability for an AWS account. I get report in CSV as shown below. The actual report has more products but this demonstrates the problem.
year_month, service_name, total_adjusted_amortized_cost
2021-01-01, Amazon ElastiCache, 100
2021-01-01, Amazon Elastic Compute Cloud, 120
2021-02-01, Amazon ElastiCache, 140
2021-02-01, Amazon Elastic Compute Cloud, 130
2021-03-01, Amazon ElastiCache, 190
2021-03-01, Amazon Elastic Compute Cloud, 120
What I would like to have is this in an excel table where the values in "service_name" are the rows and there is a column corresponding to each unique entry in "year_month". Something like:
-
2021-01-01
2021-02-01
2021-03-01
Amazon ElastiCache
100
140
190
Amazon Elastic Compute Cloud
120
130
120
So, the only solution I can think of is to use Perl or a shell script to transform the CSV file and then import it into Excel.
I am wondering if there is a better way to do this, may be using Power Query in excel. This seems to be the only way Cloudability exports the reports and I might not be the only one who has faced this issue yet.
You may want to look at pivot tables. They allow you to take data from a table and rearrange it into a new table. To create a pivot table, select the data you would like to use, then select the "Insert" tab and select "Pivot Table". You can then drag and drop the columns you would like to use into the appropriate boxes.
Related
Im quit new in ADF so here's the challenge from me.
I have a Pipeline that consist a LookUp activity and ForEach and inside this a Copy Activity
When i run this pipeline the first output of the Lookup activity looks like this
The output contains 11 different values. From my perspective i only see 11 records that will need to be copied to my Sink which is Azure SQL DB.
The input of the ForEach activity looks like this
During the running the Pipeline copy 11 times and in my sql database it has now 121 records. This amount is based on 11 rows multiple 11 iteration. This is not the output which i expected.
I only expect 11 rows in my sink table. How can i change this pipeline in order to achieve the expected outcome of only 11 rows?
Many thanks!
In order to copy data, Lookup activity and copy data source activity should not be given same configuration. If given so, duplicate rows will be copied.
I tried to repro the same in my environment.
If 3 records are there in source data, 3 times 3 records will be copied.
In order to avoid duplicates, we can use only copy activity to copy data from Source to sink.
Only 3 records are in target table.
I am running crosstabs from a survey in SPSS - Layered 2 independent variables on one question (the dependent variable). There are 1000 responses total, but whenever I run the crosstab it returns the data with 1001 or 999 or 1002 responses total. I have tried using both the Custom Tables feature and the Descriptive Crosstab feature. Neither will give me the correct totals, and all of the counts are just barely off. Is this a common problem? How do I fix it? Thanks in advance.
I need help in adding a column in this pivot table that is "Minutes Per Page".
To compute for the Minutes Per Page, it should be Total Minutes (40) divided by Total Pages (20) so in the table below, the value in the Minutes Per Page column should be 2.00 and not 3.75. 3.75 is the result of Minutes divided by Pages of each individual row and not the Total. I have exhausted my resources and still can't find the answer and so again, I turn to Stackoverflow. Thank you!
I would strongly recommend avoiding building any systems reliant on MS Access Pivot Tables as this feature is deprecated and has been removed from all versions after MS Access 2010.
As an alternative, I would suggest exporting your data to Excel (or using an external data connection from within Excel linked to your Access database), and creating the pivot table there.
You can then easily create a Calculated Field as one of the standard options offered for an Excel pivot table.
I need to create a database which allow for easy manual entry of data as well as linking to SQL databases I have access to. This is something I've been researching and trying to design by myself for a while now. Everytime I think I have a solution I begin to have doubts that this is the right direction. I would really REALLY appreciate your help/input.
My current workflow and nightmare.
I work in IT and I'm responsible for reporting for my own group but it has been and will continue to expand across many groups across the whole IT organization. Each group has their own reporting tool with some that can export data cleanly/consistently, others that have to input data manually and a few which I can access the SQL database directly but haven't yet.
To create a monthly report I have a master workbook with graphs that reference data in multiple data collection workbooks. In these data collection workbooks I have the raw data which either myself or the team/service owners inputs each month. From that raw data I may create a pivot table to get the data I want and then in each one of the collection workbooks I have a 'reference" tab. In this reference tab I have a formula that is a rolling 13 months in the first column which say goes from Jan 2013 to Jan 2014. The months are calculated by referencing a cell that has the current reporting month in it. So for this example its Jan 2014. When you change that cell to the next month say Feb 2014 it will roll that first column so that those same cells now go from Feb 2013 to Jan 2014. the columns to the right of this column reference either the data table directly or the pivot tables using vlookups() or getpivotdata() formulas that reference the month cell to the left and grabs data accordingly. Here is an example of a collection workbook.
As you can see this can be a very manual and tedious workflow with many points where this could be improved. One major issue is the fact that each team/group doesn't pull data from the same tool and doesn't serve it up in the right format. I sometimes have to write a vba script to take the raw data and add helper data like a column with the correctly formatted 02/01/2014 format so it works in pivot tables correctly instead of multiple entries with different days (ie. 02/05/2014 02/23/2014).
The end goal + where I want to be
I want to create a database where I can link SQL databases directly to it as well as making it easy for a team to manually input data by reading in a csv or xls file of data. The crux again is that I'm only one person and I want to offload as much as the data entry to each one of those teams/groups without making it a difficult task.
Using the master sheet to create a monthly report has some great results but I would like this to be a web accessible and dynamic dashboard. I'm a pretty good front end dev with knowledge of HTML, CSS+LESS and some JS+SQL knowledge. I will have support of a senior web developer if I can proceed with this project. I just need help in working out the data collection/storage/querying portion of this.
If you got this far thank you so much for reading about my problem. If I wasn't clear about something or you have more questions for me please feel free to ask.
THANKS!
You may want to take a look at SQL Server's linked servers, according to the documentation, you can link any database who have an OLE DB driver so you can run queries mixing SQL Server and other databases, so it may be worth your while to check this.
First up, this might be the wrong place to ask this question.. So, sincere apologies for that!
I have two MySQL Tables as follows:
First Table: employee_data
id name address phone_no
1 Mark Some Street 647-981-1512
2 Adam Some Street 647-981-1214
3 John Some Street 647-981-1452
Second Table: employee_wages
id employee_id wages start_date
1 3 $15 12 March 2007
2 1 $20 10 Oct 2008
3 2 $18 2 June 2006
I know, both these tables can be combined into one and there is no need to split this data into two tables. But, what i'm working on requires this data to be separate and in two different tables.
Now, previously my company used to handle all this data in Excel sheets and they followed the conventional method of having these two tables combined into one sheet as follows:
Excel Sheets
id name wages start_date
1 Mark $20 10 Oct 2008
2 Adam $18 2 June 2006
3 John $15 12 March 2007
Now, the objective is to Export the data from Excel sheets into MySQL Tables.
As you can notice employee_data.id is linked to employee_wages.employee_id
How can i replace the values in the Excel Sheet 'name' column so that they represent the actual unique ID they're given in the employee_data.id column..
May be i can do it with PHP/MySQL or i can get this done in VB Script.. BUt, I'm not an expert in VB Script..
Any help will be much appreciated..
Thanks!
Save your Excel data as a csv file (In Excel 2007 using Save As)
Check the saved file using a text editor such as Notepad to see what it actually looks like, i.e. what delimiter was used etc.
Start the MySQL Command Prompt (I’m lazy so I usually do this from the MySQL Query Browser – Tools – MySQL Command Line Client to avoid having to enter username and password etc.)
Enter this command:
LOAD DATA LOCAL INFILE ‘C:\\temp\\yourfile.csv’ INTO TABLE database.table FIELDS TERMINATED BY ‘;’ ENCLOSED BY ‘”‘ LINES TERMINATED BY ‘\r\n’ (field1, field2);
[Edit: Make sure to check your single quotes (') and double quotes (") if you copy and paste this code - it seems WordPress is changing them into some similar but different characters]
Done!
use the vlookup function in excel.
import the data into excel
use vlookup to combine what you need
You can query MySQL from Excel, this example uses INSERT: Excel VBA: writing to mysql database
A recordset can be written to Excel with CopyRecordset : http://support.microsoft.com/kb/246335