Is it possible to search for something that is in two databases? For example, I want to do a "starts with" search on a column in Postgres as well as a column in MySQL where one is "name" and one is "email"
Copying over data is not reliable as new data will be created in both databases constantly.
Yes, it is possible. For the "starts with" part, you should be able to use the standard Postgres string functions, of which starts_with is one, and indexing on the desired columns.
Getting the data from MySQL is the more complicated part.
You would most likely want to use a foreign data wrapper (e.g. FDW) from Postgres to access the MySQL data, and then handle the unioning of it (or other desired processing) with the Postgres data for returning the combined data set.
You could write your own FDW if you have particularly specific requirements, or you could try an open source one, such as this one from EnterpriseDB. EnterpriseDB is a Postgres consultancy and offers their own Postgres version, but the doc on the Github page for this says it is compatible with base Postgres as well as their own version.
Related
My Application's data is stored in both MongoDB and MYSQL. Is there any way to write queries in Metabase that will help in fetching data from both MYSQL and Mongo Databases?
I'm assuming here that by "My application's data" you are refering to the data sources, not to the Metabase internal databasa (aka metadata).
As of now (0.32.10), there is no way to query data from two different data sources.
What you can do, though, is to setup two (or more, depending on your need) different questions, and add them to a dashboard - that way, you will be able to show the data in a "data source agnostic" way.
If I didn't not get it wrong, I think this will be somewhat possible in the incoming version (0.33), scheduled for the next weeks (I believe), using the new join system (which will allow you to join data from different data sources, given a certain common key - say, "order_id" or something like that).
You can get more info, and even test the new version (which is currently in the RC2) at
this link.
My use case is to take data from one version of MySQL schema and put it in another. So even before putting the data, I want to check if the schema of the source is compatible with destination. For examples, if the new column is added in the destination and which is nullable, then they are still compatible, where as dropping a column is not compatible change since source now has extra column and destination doesn't and will break import of data.
to compare the schema of two MySQL databases I suggest to use:
TiCodeX SQL Schema Compare (https://www.ticodex.com).
It also gives you the migration script to update the destination database in case there are differences.
It's a very cheap but professional tool and with the same license you can use it also for MicrosoftSQL and PostgreSQL databases.
It's worth to mention that is the only tool I've found that also works nicely on Linux and MacOS.
We are looking at ways of improving "search" functionality in our large business application which currently uses SQL Like syntax to do it. So we started evaluating Solr server and were able to index few of our database tables and search. But I am newbie and wanted to know if
1) We have large number of tables in our application. Is there any utility that generates schema xml in solr using the database tables?
2) Our current search lists the database row that meets the search criteria (this was written using SQL 'like' and takes lot of time to generate the search results). We want to simulate the exact functionality using solr. Is that possible?
For importing a database into SOLR, you might want to look into DataImportHandler.
There will be a fair amount of configuration required for it, defining what tables and columns to import, what should be stored, and how it should be indexed.
I got indexed a Mysql database using Solr and everything is perfect. Now i got another database which uses exactly the same schema as my first database but with different data in it.
What i want is to use Solr to index also the second database using the same solr schema that i created for my first database since are completely the same!
I read that Solr cores allows you to run multiple instances that use different configuration sets and indexes, but in my case i got the same exactly configuration, the only thing that changes is the database name.
My question is what is the best way two create two Solr instances that use the same configuration?
Cheers
You could use two cores and share a schema. Just read the Wiki. But in practice you might want to keep the flexibility and just copy the schema for a second core.
How about using only one solr instance but have a field in the schema that contains a value which indicates which db/source the record came from.
I have a dataset with a lot of columns I want to import into a MySQL database, so I want to be able to create tables without specifying the column headers by hand. Rather I want to supply a filename with the column labels in it to (presumably) the MySQL CREATE TABLE command. I'm using standard MySQL Query Browser tools in Ubuntu, but I didn't see in option for this in the create table dialog, nor could I figure out how to write a query to do this from the CREATE TABLE documentation page. But there must be a way...
A CREATE TABLE statement includes more than just column names
Table name*
Column names*
Column data types*
Column constraints, like NOT NULL
Column options, like DEFAULT, character set
Table constraints, like PRIMARY KEY* and FOREIGN KEY
Indexes
Table options, like storage engine, default character set
* mandatory
You can't get all this just from a list of column names. You should write the CREATE TABLE statement yourself.
Re your comment: Many software development frameworks support ways to declare tables without using SQL DDL. E.g. Hibernate uses XML files. YAML is supported by Rails ActiveRecord, PHP Doctrine and Perl's SQLFairy. There are probably other tools that use other format such as JSON, but I don't know one offhand.
But eventually, all these "simplified" interfaces are no less complex to learn as SQL, while failing to represent exactly what SQL does. See also The Law of Leaky Abstractions.
Check out SQLFairy, because that tool might already convert from files to SQL in a way that can help you. And FWIW MySQL Query Browser (or under its current name, MySQL Workbench) can read SQL files. So you probably don't have to copy & paste manually.