NestJS + MySQL: how to connect to multiple databases without setting entities - mysql

The NestJS documentation about Databases explains how to connect to MySQL using TypeORM and defining Entities.
In my case, I have to connect to an MySQL server that has more than one database and I need to execute SQL queries directly (without using the Entity layer) and fetch their results. Sometimes I also need to run cross-database queries.
How to do that using NestJS?

You can import multiple databases by given the connections different names. You can either directly pass the database configuration in separate TypeOrmModule.forRoot({...}) imports or use a ormconfig.json config file. (However, the ormconfig.json file might still not work with multiple databases, see this thread.)
TypeOrmModule.forRoot({
...defaultOptions,
name: 'personsConnection',
^^^^^^^^^^^^^^^^^^^^^^^^^^
host: 'person_db_host',
entities: [Person],
}),
TypeOrmModule.forRoot({
...defaultOptions,
name: 'albumsConnection',
^^^^^^^^^^^^^^^^^^^^^^^^^
host: 'album_db_host',
// You can also leave the entities empty
entities: [],
})
As stated by Kamil in the comments, you can inject the TypeORM connection object with #InjectConnection('albumsConnection'): Connection and then use the QueryBuilder or the method query to run raw SQL.
const rawData = await connection.query(`SELECT * FROM USERS`);

Related

Sequelize Run Script File

I have a project which is using Sequelize to manage a set of MySQL databases. Thus far I've been able to run simple queries to create new databases, insert parameters into a table, and select data... however, I have a very long .sql file (+1,700 lines) which when executed will set up a database with a specific schema (ie. tables, views, etc.). The problem is that I can not figure out how to execute a script like this using sequelize. I know the script works on a new database because I can execute the sql file from MySQL Workbench, however I do not know how to execute the script from javascript file using sequelize. I've searched forums but can't seem to find any resources either. Can this be done?
You can run raw query by Sequelize using sequelize.query(sql_string)
and you can use fs or fs-extra to read the sql file;
Just mind that you need to set the multiline statement option true in order to run this sql text:
var sql_string = fs.readFileSync('path to file', 'utf8');
const sequelize = new Sequelize('database', 'username', 'password', {
host: 'localhost',
dialect: /* one of 'mysql' | 'mariadb' | 'postgres' | 'mssql' */,
dialectOptions: {
multipleStatements: true
}
});
sequelize.query(sql_string);
Edit 1:
To better understanding of Sequelize class take a look at this

use mysql for mailman3

I set up mailman3 using the mailman-suite configuration. I expected that mailman uses the database provided there (like sqlite or mysql), but it seems to have its own database file mailman.db in var/data. I followed the instructions here and added
[database]
class: mailman.database.mysql.MySQLDatabase
url: mysql+pymysql://myuser:mypassword#mymysqlhost/mailman?charset=utf8&use_unicode=1
to my mailman.cfg. Of course I replaced myuser, mypassword, mymysqlhost with the correct data. But still, mailman3 uses the data from mailman.db
What do I have to do to make mailman use the mysql database?

Cannot switch a connection string in LLBL Gen Pro

I have two databases with the same schema inside a Sql 2008 R2 Server, of which names are Database1 and Database2. I connected and performed queries on the Database1, and then changed to Database2 to fetch my entities using the following code
this.ConnectionString = "Server=TestServer; Database=Database2;Trusted_Connection=true";
using (IDataAccessAdapter adapter = new DataAccessAdapter(this.ConnectionString))
{
var entities = new EntityCollection<T>();
adapter.FetchEntityCollection(entities, null);
return entities;
}
(The connection string was set before executing the code).
I debugged the application and looked at the value of the connection string, it pointed to the Database2.
However, when I executed the above code, the result was return from the Database1. And if I looked at SQL Profiler, the statement was executed against Database1.
So, could anyone know what was going on? Why the query was executed against the Database1, not Database2.
PS: If I used the above connection string with plain ADO.NET, I was able to retrieve data from Database2.
Thanks in advance.
I have figured out what was going on. The reason was: by default LLBL Gen Pro uses fully qualified names like [database1].[dbo].[Customer] to access database objects, and the catalog is specified when generating entities. So you can't access objects just by changing the connection string.
Hence, to change to another database you have to override the default catalogue by using following code
var adapter= new DataAccessAdapter(ConnectionString, false,
CatalogNameUsage.ForceName, DbName)
{CommandTimeOut = TenMinutesTimeOut};
More information can be found at the following link

converting database from mysql to mongoDb

is there any easy way to change the database from mysql to mongoDB ?
or better any one suggest me good tutorial do it
is there any easy way to change the database from mysql to mongoDB ?
Method #1: export from MySQL in a CSV format and then use the mongoimport tool. However, this does not always work well in terms of handling dates of binary data.
Method #2: script the transfer in your language of choice. Basically you write a program that reads everything from MySQL one element at a time and then inserts it into MongoDB.
Method #2 is better than #1, but it is still not adequate.
MongoDB uses collections instead of tables. MongoDB does not support joins. In every database I've seen, this means that your data structure in MongoDB is different from the structure in MySQL.
Because of this, there is no "universal tool" for porting SQL to MongoDB. Your data will need to be transformed before it reaches MongoDB.
If you're using Ruby, you can also try: Mongify
It's a super simple way to transform your data from a RDBS to MongoDB without losing anything.
Mongify will read your mysql database, build a translation file for you and all you have to do is map how you want your data transformed.
It supports:
Auto updating IDs (to BSON ObjectID)
Updating referencing IDs
Type Casting values
Embedding tables into other documents
Before save filters (to allow changes to the data manually)
and much much more...
Read more about it at: http://mongify.com/getting_started.html
There is also a short 5 min video on the homepage that shows you how easy it is.
Here's what I did it with Node.js for this purpose:
var mysql = require('mysql');
var MongoClient = require('mongodb').MongoClient;
function getMysqlTables(mysqlConnection, callback) {
mysqlConnection.query("show full tables where Table_Type = 'BASE TABLE';", function(error, results, fields) {
if (error) {
callback(error);
} else {
var tables = [];
results.forEach(function (row) {
for (var key in row) {
if (row.hasOwnProperty(key)) {
if(key.startsWith('Tables_in')) {
tables.push(row[key]);
}
}
}
});
callback(null, tables);
}
});
}
function tableToCollection(mysqlConnection, tableName, mongoCollection, callback) {
var sql = 'SELECT * FROM ' + tableName + ';';
mysqlConnection.query(sql, function (error, results, fields) {
if (error) {
callback(error);
} else {
if (results.length > 0) {
mongoCollection.insertMany(results, {}, function (error) {
if (error) {
callback(error);
} else {
callback(null);
}
});
} else {
callback(null);
}
}
});
}
MongoClient.connect("mongodb://localhost:27017/importedDb", function (error, db) {
if (error) throw error;
var MysqlCon = mysql.createConnection({
host: 'localhost',
user: 'root',
password: 'root',
port: 8889,
database: 'dbToExport'
});
MysqlCon.connect();
var jobs = 0;
getMysqlTables(MysqlCon, function(error, tables) {
tables.forEach(function(table) {
var collection = db.collection(table);
++jobs;
tableToCollection(MysqlCon, table, collection, function(error) {
if (error) throw error;
--jobs;
});
})
});
// Waiting for all jobs to complete before closing databases connections.
var interval = setInterval(function() {
if(jobs<=0) {
clearInterval(interval);
console.log('done!');
db.close();
MysqlCon.end();
}
}, 300);
});
MongoVUE's free version can do this automatically for you.
It can connect to both databases and perform the import
I think one of the easiest ways is to export the MySQL database to JSON and then use mongorestore to import it to a MongoDB database.
Step 1: Export the MySQL database to JSON
Load the mysql dump file into a MySQL database if necessary
Open MySQL Workbench and connect to the MySQL database
Go to the Schema viewer > Select database > Tables > right-click on the name of the table to export
Select 'Table Data Export Wizard'
Set the file format to .json and type in a filename such as tablename.json
Note: All tables will need to be exported individually
Step 2: Import the JSON files to a MongoDB using the mongorestore command
The mongorestore command should be run from the server command line (not the mongo shell)
Note that you may need to provide the authentication details as well as the --jsonArray option, see the mongorestore docs for more information
mongoimport -d dbname -u ${MONGO_USERNAME} -p ${MONGO_PASSWORD} --authenticationDatabase admin -c collectionname --jsonArray --file tablename.json
Note: This method will not work if the original MySQL database has BLOBs/binary data.
I am kind of partial to TalendOpenStudio for those kind of migration jobs. It is an eclipse based solution to create data migration "scripts" in a visual way. I do not like visual programming, but this is a problem domain I make an exception.
Adrien Mogenet has create a MongoDBConnection plugin for mongodb.
It is probably overkill for a "simple" migration but ut is a cool tool.
Mind however, that the suggestion of Nix will probably save you time if it is a one-of migration.
You can use QCubed (http://qcu.be) framework for that. The procedure would be something like this:
Install QCubed (http://www.thetrozone.com/qcubed-installation)
Do the codegen on your database. (http://www.thetrozone.com/php-code-generation-qcubed-eliminating-sql-hassle)
Take your database offline from the rest of the world so that only one operation runs at a time.
Now write a script which will read all rows from all tables of the database and use the getJson on all objects to get the json. You can then use the data to convert to array and push it into the mongoDB!
If anyone's still looking for a solution, i found that the easiest way is to write a PHP script to connect to your SQL DB, retrieve the information you want using the usual Select statement, transform the information into JSON using the PHP JSON Encode functions and simply output your results to file or directly to MongoDB. It's actually pretty simple and straight forward, the only thing to do is to double check your output against a Json validator, you may have to use functions such as explode to replace certain characters and symbols to make it valid. I have done this before however i currently do not have the script at hand but from what i can remember it was literally half a page of code.
Oh also remember Mongo is a document store so some data mapping is required to get it to be acceptable with mongo.
For those coming to this with the same problem, you can check out this Github project. This is an ongoing development that will help you migrate data from MySQL database to MongoDB by simply running a simple command.
It will generate MongoDB Schemas in TypeScript so you can use them later in your project. Each MySQL table will be a MongoDB collection, and datatypes will be efficiently converted to their MongoDB compatibles.
The documentation for the same can be found in the project's README.md. Feel free to come in and contribute. Would like to help if need be.
If you are looking for a tool to do it for you, good luck.
My suggestion is to just pick your language of choice, and read from one and write to another.
If I could quote Matt Briggs (it solved my roblem one time):
The driver way is by FAR the most straight forward. The import/export tools are fantastic, but only if you are using them as a pair. You are in for a wild ride if your table includes dates and you try to export from the db and import into mongo.
You are lucky too, being in c#. We are using ruby, and have a 32million row table we migrated to mongo. Our ending solution was to craft an insane sql statement in postgres that output json (including some pretty kludgy things to get dates going properly) and piped the output of that query on the command line into mongoimport. It took an incredibly frustrating day to write, and is not the sort of thing that can ever really be changed.
So if you can get away with it, use ado.net with the mongo driver. If not, I wish you well :-)
(note that this is coming from a total mongo fanboi)
MySQL is very similar to other SQL databases, so I send You to the topić:
Convert SQL table to mongoDB document
You can use the following project.It requires solr like configuration file to be written.Its very simple and straight forward.
http://code.google.com/p/sql-to-mongo-importer/
Try this:
Automated conversion of MySQL dump to Mongo updates using simple r2n mappings.
https://github.com/virtimus/mysql2mongo

How can I get the database name from a Perl MySQL DBI handle?

I've connected to a MySQL database using Perl DBI. I would like to find out which database I'm connected to.
I don't think I can use:
$dbh->{Name}
because I call USE new_database and $dbh->{Name} only reports the database that I initially connected to.
Is there any trick or do I need to keep track of the database name?
Try just executing the query
select DATABASE();
From what I could find, the DBH has access to the DSN that you initially connected with, but not after you made the change. (There's probably a better way to switch databases.)
$dbh->{Name} returns the db name from your db handle.
If you connected to another db after connected with your dbh, using mysql query "USE db_name", and you did not setup a new perl DBI db handle, of course, $dbh->{Name} will return the first you previously connected to... It's not spontaneic generation.
So to get the connected db name once the db handle is set up - for DBI mysql:
sub get_dbname {
my ($dbh) = #_;
my $connected_db = $dbh->{name};
$connected_db =~ s/^dbname=([^;].*);host.*$/$1/;
return $connected_db;
}
You can ask mysql:
($dbname) = (each %{$dbh->selectrow_hashref("show tables")}) =~ /^Tables_in_(.*)/;
Update: obviously select DATABASE() is a better way to do it :)
When you create a connection object it is for a certain database. In DBI's case anyway. I I don't believe doing the SQL USE database_name will affect your connection instance at all. Maybe there is a select_db (My DBI is rusty) function for the connection object or you'll have to create a new connection to the new database for the connection instance to properly report it.
FWIW - probably not much - DBD::Informix keeps track of the current database, which can change if you do operations such as CREATE DATABASE. The $dbh->{Name} attribute is specified by the DBI spec as the name used when the handle is established. Consequently, there is an Informix-specific attribute $dbh->{ix_DatabaseName} that provides the actual current database name. See: perldoc DBD::Informix.
You could consider requesting the maintainer(s) of DBD::MySQL add a similar attribute.