Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I'm coping MySQL databases to Red-Shift with the help of an ETL tool called Matillion, and I'm using the same tool to query the database. Most of the queries I've written are basic select queries with lots of joins, unions and sub-queries.
Since Red-Shift is specialized for analytical processing, I want to transform my basic queries into OLTP queries.
I'll be grateful if someone could point me a direction to learn how to write queries more OLTP way.
Thanks!
To clarify, Redshift is not an OLAP database (like HANA or SSAS), so you can't query Redshift in an OLAP way.
However, Redshift does of course support the full range of analytic functions, which are very much OLAP-like: http://docs.aws.amazon.com/redshift/latest/dg/c_Window_functions.html
Matillion supports that too, for example with the
Window Calculation Component
https://redshiftsupport.matillion.com/customer/portal/articles/1991935-window-calculation-component
You can also search for a Rank Component on the Matillion ETL for Amazon Redshift support portal.
Matillion also has documentation/videos on Data Quality Framework which goes through some of these.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I want to write few process intensive back-end scripts for a web application using Perl and MySQL.
I will be frequently using embedded SQL in my scripts. Can anyone suggest a structured way to do that. I feel it will take lot of time to create generic classes for tables, if I have to use tables as classes and create functions like Add, Update or find to run Insert, update or select statements. Also such code itself will be quite vast than my actual business code. On the other side I do not want to use unstructured queries spread allover the code.
Are there any inbuilt class libraries to take care of my needs.
Any suggestions are greatly appreciated.
DBIx::Class is the best in class for an object-relational mapper in Perl. It will take care of all the SQL to add, update, delete and search and will do it efficiently. You tell it the tables, columns, keys and relationships. DBIx::Class::Schema::Loader can even do that part for you.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I'm using mysql for my database needs and the total size is ~1.5 TB.
I'm looking for a Business intelligence tool which gives me an interface to fetch reports in any desired format to help decision making. The user shall be able to sum, sort, count, sum in various ways.
I don't know what shall I do about it.
In the past few days, I've read dozen of articles and watched many videos about Bigdata, hadoop, BIRT, Solr, Cassandra, mongodb, noSql and just everything that popped in search suggestions for my queries on BI but can't decide what's the right tool for me.
Is there a tool available to help me with my Data reporting and visualizing needs?
Did you read about Tableau (http://www.tableausoftware.com/)? Its a very nice data visualization tool (though not free but worth it) and can connect to various data sources (mysql, hive etc). Also it provides options to do all such operations you mentioned. It will be worth to evaluate it once.
It provides a 15 day free version. You can try out and see if it fits your needs.
Try Google Big Query, it should solve your issue
https://cloud.google.com/bigquery/
For syntax, please refer
https://cloud.google.com/bigquery/query-reference
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Closed 8 years ago.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Improve this question
I need to know whether there exists any technology to query both SQL Server and Oracle from single application and single query language.
From my limited understanding, and I could be misunderstanding your question, the technology you would use to query both MS SQL Server and Oracle would be:
SQL
...but with different connection strings:
https://www.connectionstrings.com/oracle/
https://connectionstrings.com/sql-server/
T-SQL is the procedural language for SQL Server and Sybase; PL-SQL is Oracle's procedural language.
The two databases have relational theory and SQL in common, but they're very different.
Trying to unify two relational database products into one is a fool's errand.
If i got you correctly you want to develop an application that you can switch databases successfully without much effort. If you are developing using Java you can use Hibernate JPA. It makes it very easy to switch your Application from one database to another with minimal changes and without writing a single sql query. I hope this is what you were looking for.
Hibernate(NHibernate for dotnet) or equivalent is the only choice.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I need to synchronize data between two databases. The primary database is a SQL server database where all insert, update and delete operations take place. The other database is a MySQL database that reflects the state of primary database at the time of synchronization.
Note that Real-time synchronization is not important, the synchronization will done randomly depending on operator and network availability.
My questions:
What are the possible ways to determine that the two databases are already in sync and synchronization is NOT required
What are the possible ways to push data from SQL to MySQL server (no need to pull data from MySQL)
Should I use custom scripting or is there a tool that can take care of the preocess
Try Pentaho Kettle, which is an industrial strength ETL tool. We wrote a custom Perl script to synchronize, which also works, before finding Kettle.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
What is the best FREE solution to implement one ETL project in MySql?
I need to extract for analisys big amount of data, and put the results in other tables.
Regards,
Pedro
Pentaho Kettle (PDI) is open source and it has a community version here, which works quite good.
Talend also does an excellent job for ETL and ELT. You can take a look at this page on my website: http://www.hiregion.com/2010/01/data-loading-through-talend-etl-studio.html and related articles. I have also loaded hundreds of thousands of rows to millions through MySQL bulk loading (LOAD DATA INFILE syntax - dev.mysql.com/doc/refman/5.1/en/load-data.html ) and then doing some transformations in MySQL. You can do most of transformations before the load (ETL) or after load (ELT) or use hybrid technique.