I am looking for a way to store auto-generated reports. There are about 10-15 columns and 100-3000 rows depending on the report but each report is consistent in column count.
I am looking for a way to organise and store these reports into a large group without creating an entire new database and 1000s of tables to store each indervidual report.
The reports need to be queryable so they can be subdivided by team/area/person etc as each report can be a combination of 3-4 different sub-reports depending on how you split/sort the data.
I am using Python to collect and sort the data from the database so using MariaDB/MySQL would be preferred but im happy to use something else if there is a pre-exising connection libary for it.
To sum up i need something similar to a excel spreadsheet with each table being a sheet and sheet name being the date it was generated so i can select by the date generated.
Think through the goals.
Is this a legal issue -- you need to produce an unalterable report as something "official". A la a non-editable .pdf?
(at the opposite extreme) Be able to generate (or regenerate) any report for any timeframe.
Is performance an issue? (Either perceived or real)
I like to build and maintain Summary Table(s) for any "Data Warehouse" application. And build "reports" that take as a parameter a date range and a small number of other things. And have the report generation so fast that it does not matter if multiple people are pulling reports at random times.
15 columns and 3000 rows is usually excessive. If pulling a report is trivial enough, it can be less 'massive'; just get the parts you want, without such bulk.
http://mysql.rjweb.org/doc.php/summarytables
Related
I'm attempting to create a report in Access 2016 that resembles a bank deposit slip however I've been unable to find any way to display anything more than one record per report.
I have a query "Transactions" that sorts everything out and gives a datasheet with all the transactions to be used, the goal is to populate the report with up to around 12 different values from different transactions.
Is this even possible within access? My only thought thus far is to create a table with individual columns for each transaction, but that just seems messy.
Thanks for the help
We are needing to build a huge amount of reports, but alot of the metrics and fields are duplicated. Is it possible to build 1 (or a few) report to incorporate everything, and then based on what 'report' is requested to be run by the user, it hides/shows certain fields?
e.g. master report contains columns 1-100
user 1 needs to run report A, which uses columns 1-20, so hides columns 21-100
user 2 needs to run report B, which uses columns 21-40, so hides columns 1-20 and columns 41-100
Any help would be appreciated!!
Yes it's possible.
Go to condition explorer and create a string variable based on
#sq($account.defaultName)#
Create values for all possible usernames.
Use this variable as a Style variable and set up report presentation (show/hide columns) for all values.
And think about using groups not named users with
#CSVIdentityNameList()#
It will be more complicated but you won't stick to chosen usernames.
I would create report views pointing to the single report, one for each version of the report you want to run. Each view would pass in a different static value for a common parameter, say 'reportType'. The report would then use this static value to change the output returned to the user. This can be accomplished with hiding columns as Alexey suggested or you can create multiple pages and use a render variable that tests the value of the 'reportType' parameter and renders the appropriate page. The benefits of this approach as opposed to hiding is easier maintenance and a potential performance improvement as non-displayed columns are not retrieved from the data source, especially if the non-displayed columns force expensive unnecessary joins.
I have an LDAP CSV file that is imported nightly and dumped into my MYSQL database. It has about 70000 employee records.
Included in that is empl#, email, group, supervisor, etc.
I have reports that are being generated from various web sites. We are dumping these reports in the database once a month. These reports usually have empl#, email, hits, logins, whatever...
My goal is to combine the report data and add in things like group, supervisor, etc based on empl#... Speed is a big concern because of the size of the database and number of users.
At first I thought of making a simple left join (given that report data is left - and that all people in the report may not be an employee). However the problem with that is that it does not take a snapshot in time. If report data from 6 months ago is viewed I don't want it mixed with current employee data - I want it to stay a snapshot in time.
What is the best way to handle this?
You will need a date column of some kind in both sets of data on which to join. Once you have that, you can simply put a condition that establishes the snapshot in the WHERE that limits the selection.
I have a project where establishments are inspected anything from once every 6 months to once every 3 years and the results of the inspection scorecard are recorded as a record in a type 2 slowly changing dimension table [tblInspections], using StartDate and EndDate to cover the period between inspections for which this scorecard is valid. The inspections table is linked to [tblEstablishments] which contains other details about other fixed dimensions such as location and business type.
So currently, we are providing aggregated reports of current situation (where EndDate is null) and also audit reports for the history of any one establishment (On EstablishmentID)
My next task is to provide more detailed analysis reports of trends of the scorecard results and I need to provide historical aggregated results of the situation on the last day of each month.
My problem is that despite knowing exactly what I want, I am now unsure how to get there.
1) Do I start by writing ETL process to build a cube based on all the historical results working out what all the aggregates would have been at the end of each month?
2) Am I then able to just process the current records at the end of each month effectively add a new slice onto the end of an existing cube without reprocessing from scratch? (if so how?)
3) Is there another way of doing this? Does Analysis Services have better ways of dealing with SCDs automatically when determining historical status at any point in time by selecting the correct record from multiple records with start and end date?
Any advice and pointers to tutorials related to this would be much appreciated.
First I think you are going to want to build a new periodic (monthly) snapshot fact table if you are trying to analyze the inspection results across establishments (and other dimensions, like time/date). Then you can build the ETL process to populate this new fact table. Finally, you can model the fact table as a new measure group in a new or existing cube...be sure to pay attention to the aggregation property of the measures in this new measure group...typically you don't want to sum periodic snapshot measures (think about what happens if you sum your bank account balance at the end of each month and look at it by year).
Yes, you will run your ETL at the end of each month which will had more rows to your periodic (monthly) snapshot fact table. Then you can just process the cube and you are all set.
Analysis Services handles SCD2 Dimensions quite well (assuming you are using Surrogate Keys...you are aren't you?). I think the business process that you are trying to model (Inspections)...is what is causing some confusion because it's no longer a dimension in this new analysis, it has become a fact (a periodic snapshot fact)
I'm currently in the process of coding a summary report in Access XP for a client and while for the most part I've gotten 99% of the fields to match correctly with the original report (created in Corel Paradox 4.0), I'm having an issue with a sum field where despite using the same set of data, I'm getting invalid answers.
For example, in one table, the sum of the fee field comes to 9,050 in the Paradox report. In Access, the table containing the data also comes to that same amount. Despite this, when I enter the expression in my Summary report, Access will always display the total as 9,005.
I've been trying to use the NZ function to cancel any nulls, however that hasn't helped. Changing the number format to currency, general, and other formats also hasn't been helping.
Any assistance/insights are greatly appreciated.
EDIT: As requested below are some of the expressions from my Access report. Everything up till now has strictly been Access code, but as I have SQL experience, I am open to implementing that type of code if needed (it actually has been on my mind for a bit)
=nz(Sum([Fee]))
Calculates just the total fees
=nz(Sum([Room & Meals]))+(nz(Sum([Commutter & Meals])))+(nz(Sum([Fee])))
This we use to calculate the grand total. It also comes out identical to the fee total for my dummy data since the first two fields don't have any totals added to them.
Not sure this will help, but you need to break this down to trouble-shoot.
Capture the detail data from this report and compare to the Corel Paradox data. Don't just test the totals. Maybe you can put them in Excel.
Break out the three fields individually and sum those: Sum([Room & Meals, Sum([Commutter & Meals, Sum([Fee]). Don't format. Manually add them together to test.
Are these integers or is there some rounding going on?
Are the text boxes on the report large enough to display the entire number?
1 may show you're comparing apples to oranges.