I set up mailman3 using the mailman-suite configuration. I expected that mailman uses the database provided there (like sqlite or mysql), but it seems to have its own database file mailman.db in var/data. I followed the instructions here and added
[database]
class: mailman.database.mysql.MySQLDatabase
url: mysql+pymysql://myuser:mypassword#mymysqlhost/mailman?charset=utf8&use_unicode=1
to my mailman.cfg. Of course I replaced myuser, mypassword, mymysqlhost with the correct data. But still, mailman3 uses the data from mailman.db
What do I have to do to make mailman use the mysql database?
Related
The NestJS documentation about Databases explains how to connect to MySQL using TypeORM and defining Entities.
In my case, I have to connect to an MySQL server that has more than one database and I need to execute SQL queries directly (without using the Entity layer) and fetch their results. Sometimes I also need to run cross-database queries.
How to do that using NestJS?
You can import multiple databases by given the connections different names. You can either directly pass the database configuration in separate TypeOrmModule.forRoot({...}) imports or use a ormconfig.json config file. (However, the ormconfig.json file might still not work with multiple databases, see this thread.)
TypeOrmModule.forRoot({
...defaultOptions,
name: 'personsConnection',
^^^^^^^^^^^^^^^^^^^^^^^^^^
host: 'person_db_host',
entities: [Person],
}),
TypeOrmModule.forRoot({
...defaultOptions,
name: 'albumsConnection',
^^^^^^^^^^^^^^^^^^^^^^^^^
host: 'album_db_host',
// You can also leave the entities empty
entities: [],
})
As stated by Kamil in the comments, you can inject the TypeORM connection object with #InjectConnection('albumsConnection'): Connection and then use the QueryBuilder or the method query to run raw SQL.
const rawData = await connection.query(`SELECT * FROM USERS`);
I am using jupyter & python 3.6.4 via anaconda.
I want to be able to process and store data from python to a MySQL database.
The libraries I am using to do this arepymysql and sqlalchemy.
For now, I am testing this localy with wamp (mysql version : 5.7.21), later I will apply it to a distant server.
Database creation function:
def create_raw_mysql_db(host,user,password,db_name):
conn=pymysql.connect(host=host,user=user,password=password)
conn.cursor().execute('DROP DATABASE '+db_name)
conn.cursor().execute('CREATE DATABASE '+db_name+' CHARACTER SET utf8mb4')
Function to convert a Dataframe to a relational table in MySql:
def save_raw_to_mysql_db(df,table_name,db_name,if_exists,username,password,host_ip,port):
engine = create_engine("mysql+pymysql://"+username+":#"+host_ip+":"+port+"/"+db_name+"?charset=utf8mb4")
df.to_sql(name=table_name,con=engine,if_exists=if_exists,chunksize=10000)
The execution code:
#DB info & credentials
host = "localhost"
port = "3306"
user= "root"
password= ""
db_name= "raw_data"
exade_light_tb = "exade_light"
#A simple dataframe
df = pd.DataFrame(np.random.randint(low=0, high=10, size=(5, 5)),columns=['a', 'b', 'c', 'd', 'e'])
create_raw_mysql_db(host,user,password,db_name)
save_raw_to_mysql_db(df,exade_light_tb,db_name,"replace",user,password,host,port)
The warning I receive when I run this code:
C:\Users.... : Warning: (1366, "Incorrect string value: '\x92\xE9t\xE9)' for column 'VARIABLE_VALUE' at row 481")
result = self._query(query)
From these threads: /questions/34165523/ questions/47419943 questions/2108824/, I could conclude the problem must be related to the utf8 charset, but I am using utf8mb4 to create my db and I am not using Django (which supposedly also needed to be configured according to questions/2108824/).
My questions :
How is this warning really impacting my data and its integrity?
How come even though I change charset from utf8 to utf8mb4, it
doesn't seem to solve the warning? Do I need to configure something
further? In this case, what are the parameters I should keep in mind
to apply the same configuration to my distant server?
How do I get rid of this warning?
Annex:
I've made an application in Java EE that uses Hibernate to communicate with MySQL. It works perfectly on my Windows development machine, but I have problem on debian, where the application is deployed.
When I search for keyword with Polish letters(like ł, ą, ć, ó etc,) the result is ok on Windows, but on server, where I have imported the database it does not work.
Hibernate query looks like this:
#NamedQuery(name = "Keyword.findByKeyword", query = "SELECT k FROM Keyword k WHERE k.keyword = :keyword")
and is called like this:
myEntityManager.createNamedQuery("Keyword.findByKeyword").setParameter("keyword", keyword).getSingleResult();
When I use mysql on debian via SSH and type in SELECT query manually:
SELECT * FROM keywords WHERE keyword = 'ser żółty';
it also works and return single result. Encoding and collations of tables and columns are also ok. In datasource configuration I've added
?UseUnicode=true&characterEncoding=utf8
parameters, but it also did not help. I thought that maybe there is a problem with encoding in data from request send by form, but the problem appears even if the parameter i.e. "ser żółty" is hardcoded in my repository class.
I also use Hibernate Search for indexing and the FullTextEntityManager return correct results with Polish letters.
I assume that there is some problem between Hibernate and MySQL, but I have no more ideas what could I change. Any suggestions?
Server Wildfly9.0.1, MySQL 5.6
Ok the problem was in encoding on the mysql server level. It was set to latin1 by default. To fix this follow this question Change MySQL default character set to UTF-8 in my.cnf? and edit your my.cnf file.
I'm writing a script in Groovy that needs to store (and later retrieve) data which contains czech characters (such as č, š, í, ě, ...).
I use a standard JDBC connection like
def sql = Sql.newInstance(
'jdbc:mysql://localhost/db',
'root',
'',
'com.mysql.jdbc.Driver'
);
sql.executeInsert(
'INSERT INTO ... VALUES ...', [ ... ]
);
I have downloaded the latest MySQL ZIP archive for 64bit Windows machines and extracted it on my local hard drive. I now run it (for testing purposes) via mysqld launched manually from the command line.
When I store and retrieve the data some of the czech symbols are corrupted (?s are displayed instead of them). I believe that the database works with a wrong encoding. I would prefer the script to work with UTF-8 encoded data.
I have found a lot of (mutually different) information on the internet about how to set MySQL to work with UTF-8 data. Non of them worked for me, though.
Could you please provide instructions for my specific use case?
As long as your db/table/column is set to use UTF-8, you can try changing your connection params to:
def sql = Sql.newInstance(
'jdbc:mysql://localhost/db?useUnicode=true&characterEncoding=UTF-8',
'root',
'',
'com.mysql.jdbc.Driver'
)
I've connected to a MySQL database using Perl DBI. I would like to find out which database I'm connected to.
I don't think I can use:
$dbh->{Name}
because I call USE new_database and $dbh->{Name} only reports the database that I initially connected to.
Is there any trick or do I need to keep track of the database name?
Try just executing the query
select DATABASE();
From what I could find, the DBH has access to the DSN that you initially connected with, but not after you made the change. (There's probably a better way to switch databases.)
$dbh->{Name} returns the db name from your db handle.
If you connected to another db after connected with your dbh, using mysql query "USE db_name", and you did not setup a new perl DBI db handle, of course, $dbh->{Name} will return the first you previously connected to... It's not spontaneic generation.
So to get the connected db name once the db handle is set up - for DBI mysql:
sub get_dbname {
my ($dbh) = #_;
my $connected_db = $dbh->{name};
$connected_db =~ s/^dbname=([^;].*);host.*$/$1/;
return $connected_db;
}
You can ask mysql:
($dbname) = (each %{$dbh->selectrow_hashref("show tables")}) =~ /^Tables_in_(.*)/;
Update: obviously select DATABASE() is a better way to do it :)
When you create a connection object it is for a certain database. In DBI's case anyway. I I don't believe doing the SQL USE database_name will affect your connection instance at all. Maybe there is a select_db (My DBI is rusty) function for the connection object or you'll have to create a new connection to the new database for the connection instance to properly report it.
FWIW - probably not much - DBD::Informix keeps track of the current database, which can change if you do operations such as CREATE DATABASE. The $dbh->{Name} attribute is specified by the DBI spec as the name used when the handle is established. Consequently, there is an Informix-specific attribute $dbh->{ix_DatabaseName} that provides the actual current database name. See: perldoc DBD::Informix.
You could consider requesting the maintainer(s) of DBD::MySQL add a similar attribute.