correct way to automate department's reporting? - mysql

I just recently started my first job doing reporting and analytical on a project team. Currently a lot of data and reports are sitting in excel spreadsheets and in some database tables. Reports are drawn from this data in a very manual adhoc approach. It would seem that this has/is quickly becoming very problematic as the reporting requirements are increasing rapidly. The project will soon be rolled out more quickly, and our manual efforts won't suffice. Currently we are spending a lot of time manually doing etl like activities, the results are then sliced and diced in excel, and graphs and tables exported for management. After which the process has to be repeated for the next reporting cycle. We need to implement some automation in this regard.
As this is only a medium term project a full blown etl and reporting solution seems out of the question- neither time nor resources permits. Being a fresh graduate I know what has to be done, i just don't know what tools, utilities and software to implement. Google is just confusing me more. Any suggestions? Really feeling like I'm in over my head.

If number of Excel files are in tens rather than hundreds, I would suggest importing them to Access manually, and/or with bit of VBA standardise the data before importing.
Then, you can use Access to make reports.
Essentially, answer to your question boils down to this: Is It Worth the Time?

I believe you really need at least a reporting tool for automate this job, a tool that really covers most of what you need it's DBxtra, it connects directly to MySQL, exports to Excel and it can send the exported results (or a pretty designed report) to anyone by email automatically (for example every Monday at 6:00 AM), the biggest problem is that you point that your data is not fully "curated" for reporting, but if you can implement SQL logic to clean it up then you can automate almost the full processs, otherwise, you can automate the exporting to Excel, do the clean up work by hand, and automate the sending of the report to the managament guys (DBxtra can also connect to Excel directly and create reports from it).
If you want to do it by hand, you can do an application on any .NET language and look up for a way to connect to MySQL (ADO.NET) and export to Excel (Excel Interop or third party library).

Related

Link TFS Export in Excel to an Access table

We do a lot of TFS analysis and reporting in Excel, using PowerPivot to map to 'executive friendly' terms (like dates vs iterations) and they are quite please with all that. However, we also now have several TFS 'projects' (instances, versions, whathaveyou), and an Excel workbook cannot link to more than one project at a time. Also as the projects grow larger and larger, my machine struggles to process. I'd like to put the mapping tables AND the TFS data from several instances into an Access database.
Question 1
Is there any way to link Access to TFS? If not, I'm fine having TFS data in Excel and linking to Excel, but with the 'header' TFS insists on putting in your export, the linked table has issues (like not having headers in first row, and then always having two erroneous records upon every refresh).
Question 2
Any thoughts on how to get around the funky header?
There is no native way to link Ms Access to TFS or to easily get rid of the header row that's added in Excel when linking a table. I'm afraid that the direction you're seeking will not help you in the end. It will only put all the computational pressure on Access instead of Excel.
The way to go here would be to setup the TFS reporting features, which offloads a lot of the processing to SQL Server Analysis Services (which can be used as a source for Excel) and reporting is provided by SQL Server Report Server in that case.
Unfortunately, the Reporting part of TFS hasn't had a decent update since 2010 and can be a bit archaic to use. For VSTS (the cloud-based version of TFS) it's now possible to link the account to PowerBI, which does everything you're after. This feature is not available on-premise though, it may be a great reason to move to the cloud. PowerBI can handle large amounts of data, can be connected to one or more VSTS accounts and can slice and dice data from multiple accounts and projects with ease.

Know of a good migration plan for SSRS 2005 reports?

Traditionally, our plucky team of developers have always migrated our own reports into production. This task is now being taken up by our support staff, who'll need detailed instructions to do so.
We have always deployed straight from VS2005, which can be made available to the new migrators. Is there a better way to migrate these reports without VS2005?
Also, we need to provide some backup and rollback instructions. Have to confess, SSRS reports are probably the only thing I haven't backed up prior to a migration ( I normally take the "hey, they're only reports!" view ). Does anyone know of a good resource which covers backup of reports prior to deployment of replacements?
Have a look at this, it will backup all your reports using a script. I've used for migrating reports between servers. If you open the script file that it generates, you'll get a good idea how to generate your won scripts that you run using the RS.exe utility built into SSRS.
http://www.sqldbatips.com/showarticle.asp?ID=62

How can I report on files with pending changes in TFS?

I'd like to create a simple report that shows files that currently have pending changes (checked out) from a TFS 2008 server. I know that I can use the "Find in Source Control" option from Team Explorer, but I would rather generate a reporting services report. Ideally, I'd be able to show when the file was checked out and the user that checked it out, but that's not imperative.
If the data isn't pushed to the TFS data warehouse by default, then I'd like to find the relational table(s) in the SQL Server instance that would need to be queried.
I've spent some time digging around the TFS data warehouse and looking at all of the canned Reporting Services reports that I can get my hands on, but everything seems to be geared towards work items, check-ins associated with work items, etc...
If you're looking for some easy to read data and not too worried about print outs, have a look at the TFS sidekick application by Attrice. Very helpful and if you have the correct permissions, you'll be able to see all the checked out files.
http://www.attrice.info/cm/tfs/
I doubt the information you're looking for is in the data warehouse and even if it was it might not be fresh enough for your purposes. By default the warehouse is updated once an hour.
You could use SSRS to report directly against the TFSVersionControl database but I would not recommend going this route. The database is not documented and chances are very good that it will change in the next version. It could also have performance implications if your queries are not written correctly.
A better solution would be to use the TFS web services as your SSRS data source. There are services you can call to get all files that are checked out. This iformation is always current and the queries it runs are highly optimized.
Example command line (Studio 2008):
"C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\tf.exe" status /recursive /user:*

Should I use SQL Reporting Services 2008 for my reporting engine?

I would like to use SQL Reporting Services 2008 to generate my reports, but I want to use my own UI for specifying the report type, columns, parameters and everything. I want to be able to take these criteria, and then kick off an asynchronous request to SSRS and have the report emailed to me. Is this possible? I don't want to go all the way down the road of researching SQL Reporting Services 2008 only to find that it doesn't do what I need it to do. Also, I will have a ton of DB partitions that the data will need to be pulled from. Some reports will need to pull data from only one of these, but other ones may actually need to span different databases. Is it possible when sending a report request to SSRS to specify what servername/database to pull the data from? Is it possible to tell it to take the data from multiple databases and combine it? Thanks.
Like Crystal Reports, ActiveReports and other report generators, SSRS has two basic elements behind each report: the SQL query and the report layout. No matter what tool you use for the SQL -- it can be inline SQL in the report or a call to a stored procedure -- it's going to be the same query. Multiple databases are fine as long as you can specify them up front.
You can have parameterized queries, so the user is prompted to input the relevant filters (customer ID, product group, date range, whatever).
Doing the report layout is similar to other tools -- you drag and drop controls like labels onto the report, and set their formatting.
SSRS does provide a lot of options for distributing the report, including email. You can embed the report in an ASP.Net web page, leave it on the report server site for users to browse to, run it in the wee hours of the morning and cache it so every user doesn't have to wait for the lengthy query to run.
It's a great tool. I think it will be worth your effort to experiment with it. I would wait on creating the customized UI until you've exhausted the possibilities inherent in the tool.
SSRS is not designed with this scenario in mind, for that matter I am not sure that any out of the box reporting solution is going to have an elegant solution for this. While SSRS can do what you are asking (as well as others), it is by no means quick or easy. You seem to be looking for an advanced ad-hoc solution with dynamic sourcing of the data. I would first question the requirements and determine if the business scenario really justifies such an implementation. I would weigh custom building a solution vs your learning curve with a BI reporting solution. You may find that it is easier to just build something on your own.
I think the heterogeneous dynamic database mashup is probably going to be the most challenging part.
Depending on what your scalability requirements are, one place that has that part covered, and a report writer, is Access. (Duck! Incoming!)
I think you may be creating a rod for your own back to a certain extent as RS ships with a few interfaces for report creation.
Mind you the end product is an rdl file which is nothing but xml, so you can write them by hand if you really like.
Multiple data sources are supported, but combining them on a single control/chart/etc are not, so you'll need to configure yourself a cross database capability from one of your data-sources prior to the report request if you want to do that.

Compare SQL Server Reporting Services to Crystal Reports [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
Which of Crystal Reports and SSRS (SQL Server Reporting Services) is better to use?
On the one-hand, Crystal Reports is a steaming pile of expensive and overhyped donkey poo, and on the other hand SSRS actually fulfils all the promises that CR marketing makes - and it's free.
My contempt for CR stems from many years of being obliged to use the horrible thing. There's really no point in detailing the utter odiousness of CR when I can give you references like Clubbing the Crystal Dodo or Crystal Reports Sucks Donkey Dork (not as funny but rather more literate and substantiated with technical details).
Free?! Yup. You don't even have to buy MS SQL Server to get it - you can install SQL Express with Advanced Services. This is available as a download that includes SQL Server Reporting Services. While SQL Express is limited in the number of concurrent users it can support, the following observations are salient:
The licence for SSRS obtained as
part of SQL Express only requires
that it be deployed as part of SQL
Express. There is nothing forbidding
connection to other data sources or
requiring that a report obtain data
from SQL Server.
The abovementioned version of SSRS
has no intrinsic restrictions on
user connections. All limitations
are imposed on the SQL Express
database engine.
SSRS uses ADO.NET, which includes,
out of the box, drivers for Oracle,
Jet (Access), OLEDB and ODBC
Thus you can connect the free version of SSRS to any back-end to which you can connect ADO.NET, which includes (for example) MySQL. I am told by Rory in a comment below that this is "not supported". That's true but I can't find anything in the licence that forbids it and while the drivers are not supplied by SSExpress they certainly are supplied by most versions of Visual Studio and you can ship them in your setup kit. This may not be an expressly supported configuration but so what? Even if you did have a full MSSQL licence it would be asking a bit much to expect Microsoft to help you talk to some third party database (not to mention a bit weird).
I use SSRS extensively at work both for inward facing reports and for outward facing reports embedded in ASP.NET applications that provide bureau services to large numbers of paying customers. In our case it happens that the backing store is a licensed copy of Microsoft SQL Server 2008, but this is incidental to the technical merits of our reporting solution.
There is a long list of capabilities that Crystal Reports claims to support but which either don't work or which require a staggeringly expensive licence if you want more than five users. You can't even trust CR to do SQL correctly. SELECT COUNT(*) FROM SOMETABLE WHERE 1=0 should produce a result of zero but it it produces one. The built-in query engine is defective, and a team that screws up something a bunch of amateurs can do for free (eg MySQL) has no hope of getting anything you'd describe as performance out of their code.
And they don't. The evil thing leaks memory like a bucket with no bottom, and if you use SQL profiling tools you will find it is spectacularly inefficient.
As for the alleged support, I can personally attest that dialog resize bugs have gone uncorrected for decades after they were first publicly documented. If you get out your credit card and pay the extortionate ransoms demanded (I too would want handsome pay to support such a horror) you will find yourself talking to someone who claims his name is David, but inexplicably pronounces it "Dah-feet", and who doesn't even understand your question, much less have an answer.
The SSRS support situation is fairly similar, but it actually works so you don't really need much.
SSRS, on the other hand, does everything that CR claims to. It is not without bugs, but they are delightfully few, and they seldom survive more than one release cycle.
The SSRS designer UI is hosted within the Visual Studio IDE. It is attractively presented in typical Microsoft style, but more than this it is quite well thought out, incorporating several simple but fundamental departures from traditional report designers. For example, to present tabular data you define a table rather than fiddling about with individual text boxes. As a result you don't have to screw around trying to line them up, and putting borders on them is a trivial stylesheet exercise.
SSRS actually does all the things CR claims to, it's inexpensive, there is extensive reliable technical documentation, it's designed to be extended (also documented) and you can connect it to anything for which you can get an ODBC driver. This is a no brainer.
Some shortcomings of SSRS
It is not obvious how to bind fields in page headers and footers.
It is not possible (so far as I know) to position relative to the bottom of a page. This is a genuine problem for certain types of report, and one for which I can think of no workaround.
There's no support for expando horizontal rollups in cross-tabulations.
There's no direct support for report headers and footers. Use Rectangle objects at top and bottom of the report layout, with pagebreaking properties set appropriately. Or use subreports. The people who complain about this obviously haven't tried very hard.
Lack of support for overlapping group intervals (the CR grouping system can do this) UPDATE SSRS 2008 R2 now supports this. It's buried in the grouping edit dialog. Look up "group variables" and read this.
It actually looks like overlapping groups can be done with SSRS2005 too, although I never knew that. I wonder did anyone ever crack the bottom-relative positioning issue?
I've been using Crystal report till version 10 and was always doing stuff i wanted successfully along with ASP.NET applications. Its output on web is really good like WYSIWYG and exports to Excel and PDF are also accurate. Printing is also marvellously correct.
Recently, i've been working on SSRS 2005 for around an year and have been living to witness so many lackings that must have been provided out-of-the-box too. SSRS web output varies greatly with different browsers and diff resolutions and would easily make a developer sick. Moreover, the scrolling issues with report viewer would make an end-user mad quite early as it is based on HTML using an IFRAME. (Note: Crystal 13 uses an IFRAME in the web-viewer which suffers from sporadic text-wrapping and overlapping issues). The exports are not good at all. You cannot align images left or center in cells and cannot specify background colors for images. You cannot center align complete report body. For possibility, i've played with the rendered HTML for hours and figured out exact replacements to make that works, but these simple fixes were not known to SSRS developers i guess because probably, they never used SSRS for themselves.
Further, in web applications, you need to bear the bad UI for parameters out-of-the-box. I have simply removed it completely and the cost of creating it in ASPX pages made me think to design tabular reports in DataGrids instead using ObjectDataSource and database pagination technique. You cannot layout the parameters to your needs. Bugs in paramters sections postsback complete reports without any changes. Paging with grouping works with a trick, but than sorting fails on complete dataset. For every bit of medium to advanced level of UI requirement, SSRS costs so much of time figuring out that it is simply not possible. As there are less of SSRS users, online community has no good solutions for simple problems. Not to forget the good side of SSRS is its deployment, notifications built-in, caching and configuration side, but no UI to win.
BOTTOMLINE is that i've seen SSRS frustating you just due to the nonresponsiveness of Microsoft Support team when they have to say 'sorry! not now' after a month. SSRS 2008 also doesn't have many of these issues fixed rightaway. Moreover, moving to SSRS' 08 means a complete migration of back-end platforms as well. Keeping the equation in mind that the more you use a software, the more it gets mature over time, Crystal is anyways a much better choice because, SSRS soon accumulates costs for fixing their bugs by yourselve.
You can deploy an app using Reporting Services by including 3 DLL files. That's a huge benefit. (Note--you have to get one of the 3 DLL files from the GAC.)
With Crystal Reports, you have to install the runtime on each machine that will run the application (either a website or client app).
Reporting Services has all of the features most people need, and the deployment is MUCH easier. I will never user Crystal Reports unless I have to.
Since this thread has popped back open, I'll add my two cents. I had to use Crystal for about three years during the version 7 and 8 days. I hated every minute of it. I've seen a little bit of the newer versions and still don't like it.
I dislike it so much that it pains me to say this: from my experience Crystal's better suited than SSRS for complex reports. A coworker and I tried desperately to get a moderately complex report layout to work in SSRS and gave up. My impression of the product -- just my opinion, mind you -- is that it's not quite ready for prime time.
Crystal will make you hate your life and look for another job, but there's a reason it's so pervasive: it works.
Reporting Services is much better in my experience. It is a better environment, but best of all the connections (data sources) are separate from the report and can be shared. This makes for much simpler deployment between environments.
I've used both, I'll add a couple of points to what's already been said:
For simple stuff, I'd recommend SSRS by default. Crystal is a bit bloated and quirky.
Crystal can easily export to MS Word format (.doc). Customers want this pretty often in my experience.
If formatting is important, Crystal may be better. For example, SSRS reports can't have more than one type of text in a single text box. Meaning that you can't have, say, a comment at the top of the report that has both italics and normal text. Crystal can do this:
Note: This report contains data from start date to end date inclusive of those dates.
SRSS can't (without multiple overlapping textboxes). I once had a 20 page word document given to me, to be converted to a report with data for the dozen or so graphs and tables in it. I started out in SSRS, but realised that in Crystal I could just copy and paste the hardcoded bits of the report straight from word, with coloured headings and all, and saved days of work. So Crystal does have a better "designer" in many respects.
Update:
Apparently both of these issues have been fixed in the current SRSS. Anyone care to comment further on this?
I agree with #Carlton partly for the reasons he describes. I also think that reporting services is a more mature product (even though Crystal Reports has been around longer). The Test and deploy model is pretty hearty, and the built-in ability to track report usage is very helpful.
I also find it much easier to design reports in Reporting Services - Microsoft has learned how to build a good IDE, whereas the Crystal IDE has always seemed like an after thought (though that's better than an afterbirth, which is what it used to be).
Edit: Additional thoughts
I also think that in a Windows shop, SSRS offers all kinds of sweet integrations with the OS and SQL Server. You can rely on SQL assemblies for built-in code reuse fairly easily in SSRS, and the integration with the Active Directory security model makes securing your reports very easy.
Man...my company has sooo many crystal reports...and the company before that had lots too. From version 8.5 to 11.5. They kind of already have their foot in the door so to speak. I think the CrystalReportViewer is a steaming piece of crap but it does work(for the most part).
After reading some of these answers, I'm switching to SSRS for my next reporting project! The writing is on the wall...MS will drop Crystal from VS and replace with SSRS. The only thing that's going to suck is when MS starts charging for it.
EDIT: Messing around with SSRS today and it looks quite promising. I must say the designer is taking some getting used to...CR Designer has it beat in ease of use. You can tell this is designed for programmers where as CR is geared toward report designers.
EDIT2: SSRS really fails to meet my reporting needs. Designing reports sucks when you want to preview and no parameter prompting available for standalone. Is there a better way to design them...preferably not in VS?
Did you think about an alternative? If you want to use the features of Crystal Reports but don't want to pay so much for it you could have a look at Crystal-Clear which is an Java based reporting tool supporting Crystal Reports templates too. It comes with a GUI-designer and data sources are also configurable per system. (Almost ODBC-like, you just set a name for the connection and the connection is configured on the system.)
I wonder why no-one mentioned one big issue with CR - that it just fails in source control or team environment. Correct me if I am wrong but I really looked very hard for any report diff tools. There's one (released about year ago) but it just doesn't do well - not because it's bad but (I guess) because CR just don't expose report structure correctly or something... I tried to export .rpt to XML but it's clunky and wrong. I even tried to write my own .rpt comparer.
It's not about team development only; even if there's single developer it's a nightmare to maintain reports versions, and if your customer decided to add few things or to change few colors, you're now cursed to track every single textbox since there's absolutely no way to find out the changes.
RDL format is much more clean and open. And this can be a pretty major advantage.
I have used both for years.
Crystal reports charges way too much and I try to use SSRS whenever possible.
However, SSRS does not support firefox or any other browser, only IE, this is a problem.
The reports in Crystal look nicer and the exports are more powerful, users want good exporting to Word.
If you are a java programmer, I would use Jasper Reports, it is free and uses Java language for functions.
I've used both (Crystal Reports 2008 and SSRS 2008) because I did not notice this thread in time.
Apart from the setup which was a bit easier with CR, I could not notice a single feature where CR is at least on par with SSRS. Yes, Crystal Reports is really that bad.
In my opinion the absoultely worst part in CR is the IDE. But there are also other killer features, such as poor SQL performance and horribly looking graphs (at least in the CR version that comes with VS 2008) are also notable "killer" features.
I have worked with both CR and SSRS and this what i found.
Crystal Reports runs in its own memory while SSRS runs in the limited SQL Server memory.
Crystal report is way too expensive. Recently they have slashed their price to 250$ i think as a response to SSRS 2008 release.
SSRS is free.
The biggest reason why Crystal report thrives :
You can design 80% of reports in a project using SSRS. But for the remaining 20% you have to use some other reporting tool. These 20% reports are used by none other than top level managers , directors & CEO. Their requirement can never be undermined and CR does a wonderfull job there.
Crystal report is still COM based. which is a pain in the a**.
Crystal report is not lacking capabilities or features. It is the work horse of SAP. But lot of its classes are protected and dont provide access to programmers. This is by intention. The SAP people are so greedy they want to keep every feature under control and charge extra fortune for exposing the claases and objects to the developers under special license arrangement. Just debug and quick watch the ReportDocument object in VS you will know inspite of everything available in the object you can hardly use them in your code !!
As far as GUI & CSS issues are concerned expecting a COM object which is designed for precision printing , to render correctly in every browser is a moot point as even a simple div renders differently in different browsers.
I have been working with Crystal reports since 7 years and cursing it all the time while actively exploring all other alternatives. But i am yet to come across something as flexible as Cystal Report. For bulk of the work SSRS is good. But for Dashboards , Complex Reports with subreports, Balance sheets, trial balances i shall never waste my time in SSRS.
Just try a Google Trend search on Crystal Report. It has been steadily declining since last 6 years. surely the future does not look good for CR.
But Hey ! MS, SAP and ORACLE still endorse Crystal Report at the core of their applications !! and no BI product comes cheap.
I feel like a martian having an extensive and positive (but sometimes complex) experience with Crystal Reports, which is now completely integrated in our user interface (VBA), where requested reports parameters and filters are transparently inherited from the user interface ...
If you're considering SSRS and are concerned about the fact that it's "free" but you need to either buy and additional SQL Server license or distribute SQL Express, then you might be interested in Data Dynamics Reports
It offers all that is in SSRS and adds Master Reports, Themes, Calendar data region, Data Visualization (Databar, Sparkline, Iconset, ColorScale, ...), complete object model for maximum programming flexibility, royalty free end user report designer, barcode report item, excel template export and data merging, and much more. You can download a trial from Data Dynamics (now GrapeCity) and try it with few reports, you will not be disappointed.
I've worked with both now and have seen them side by side. Crystal has been good, but expensice over the years. Its clunky, but we've gron accustomed to it and familiar with the interface. I don't work in the LAMP environment, This house works with MS Dynamics and MAS with some pretty large clients.
I love not having to worry about the client install for SSRS. Distributions is far easier and sharing data sources and report models is working out well.
AS far as broweser go, I've seen perfectly rendered SSRS 2008 gauges in Firefox. I have exported those gauges to Excel without issue. I have deployed reports with and without MOSS to phones. The ability to use windows authentication to deploy reports as well as hide them is fantastic. The report viewer object in VS 2005 and later is sweet.
People, please refer to version of which you are talking about!
For example, the VS2008 built-in free RDLC reporting (the same as SQL Server 2005 Reporting Services) doesn't support binding fields in header and footer, and it is a basic feature!
Now I'm converting a huge report from this VS2008 Reporting / RDLC 2005 to Crystal Report 2008 Basic (which comes with VS2008) because it doesn't have this basic feature.
I am confident that Reporting Services 2.0 / RDLC 2008 (which comes with Visual Studio 2010) and better yet, the newest Reporting Services 3.0 / RDLC 2010 (which comes for FREE in SQL Server 2008 R2 Express With Advanced Services) are better SSRS solutions.
SQL Server R2 Express with Advanced Services (FREE)
http://www.microsoft.com/express/Database/InstallOptions.aspx
Right now I am making a Proof of Concept for Reporting Services 3.0 / RDLC 2010, and will post the results.
Reporting Services (SSRS/RDLC) is always more easy to work, but easy comes with a price. For simple reports, always choose SSRS/RDLC. For complex reports with master-detail, page control and so on, please make a PoC of these scenarios with newest SSRS/RDLC versions (2008,2010) and also with Crystal Reports.
For those who are comparing the old Crystal Reports XI and Reporting Service 1.0 please see this 2005 post:
SQL Server Reporting Services and Crystal Reports: A Competitive Analysis
http://www.crystalreportsbook.com/SSRSandCR_Conclusion.asp