Is it possible to set connection collation within MySql connection string and how, since there's a default setting on the server that's used for new connections.
Two things I can't do:
Can't call SET COLLATION_CONNECTION after I open a connection, because I'm using Entity Framework that does all the calls for me not entirely true as you may see in the edit
Can't change server default connection collation because of other databases and their respected applications that use them.
All I'd like to specify is a certain connection string parameter in my web.config file like:
"User id=dbuser;Password=dbpass;Host=dbserver;Database=testung;Collation=utf8_general_ci"
but Collation setting/variable isn't recognised.
Technologies used
Asp.net MVC 2
IIS 7
Entity Framework 1
DevArt dotConnect MySql connector
MySql 5.1
EDIT 1
I've tried this code as #Devart suggested but to no avail:
partial void OnContextCreated()
{
System.Data.Common.DbCommand command = this.Connection.CreateCommand();
command.CommandText = "set collation_connection = utf8_slovenian_ci;";
command.CommandType = System.Data.CommandType.Text;
this.Connection.Open();
command.ExecuteNonQuery();
// this.Connection.Close();
}
We recommend you to implement the OnContextCreated partial method.
You have access to the store connection in it and you can execute ADO.NET command "SET COLLATION = ..." using this connection.
If anyone else stumbles over this problem or wants to issue a command when opening a connection: The answer regarding OnContextCreated does no longer work as the method does no longer exist/is no longer supported.
An alternative, which I use for executing SET NAMES <character set used by the database> is to append ;initialization command=\"SET NAMES '" + CharSet + "';\" to your connection string. According to Devart's documentation this also works for PostgreSQL, MSSQL and Oracle
This property can also be set inside EntityDeveloper when accessing the properties of the database connection and clicking on the Advanced button.
Related
I am using jupyter & python 3.6.4 via anaconda.
I want to be able to process and store data from python to a MySQL database.
The libraries I am using to do this arepymysql and sqlalchemy.
For now, I am testing this localy with wamp (mysql version : 5.7.21), later I will apply it to a distant server.
Database creation function:
def create_raw_mysql_db(host,user,password,db_name):
conn=pymysql.connect(host=host,user=user,password=password)
conn.cursor().execute('DROP DATABASE '+db_name)
conn.cursor().execute('CREATE DATABASE '+db_name+' CHARACTER SET utf8mb4')
Function to convert a Dataframe to a relational table in MySql:
def save_raw_to_mysql_db(df,table_name,db_name,if_exists,username,password,host_ip,port):
engine = create_engine("mysql+pymysql://"+username+":#"+host_ip+":"+port+"/"+db_name+"?charset=utf8mb4")
df.to_sql(name=table_name,con=engine,if_exists=if_exists,chunksize=10000)
The execution code:
#DB info & credentials
host = "localhost"
port = "3306"
user= "root"
password= ""
db_name= "raw_data"
exade_light_tb = "exade_light"
#A simple dataframe
df = pd.DataFrame(np.random.randint(low=0, high=10, size=(5, 5)),columns=['a', 'b', 'c', 'd', 'e'])
create_raw_mysql_db(host,user,password,db_name)
save_raw_to_mysql_db(df,exade_light_tb,db_name,"replace",user,password,host,port)
The warning I receive when I run this code:
C:\Users.... : Warning: (1366, "Incorrect string value: '\x92\xE9t\xE9)' for column 'VARIABLE_VALUE' at row 481")
result = self._query(query)
From these threads: /questions/34165523/ questions/47419943 questions/2108824/, I could conclude the problem must be related to the utf8 charset, but I am using utf8mb4 to create my db and I am not using Django (which supposedly also needed to be configured according to questions/2108824/).
My questions :
How is this warning really impacting my data and its integrity?
How come even though I change charset from utf8 to utf8mb4, it
doesn't seem to solve the warning? Do I need to configure something
further? In this case, what are the parameters I should keep in mind
to apply the same configuration to my distant server?
How do I get rid of this warning?
Annex:
I've made an application in Java EE that uses Hibernate to communicate with MySQL. It works perfectly on my Windows development machine, but I have problem on debian, where the application is deployed.
When I search for keyword with Polish letters(like ł, ą, ć, ó etc,) the result is ok on Windows, but on server, where I have imported the database it does not work.
Hibernate query looks like this:
#NamedQuery(name = "Keyword.findByKeyword", query = "SELECT k FROM Keyword k WHERE k.keyword = :keyword")
and is called like this:
myEntityManager.createNamedQuery("Keyword.findByKeyword").setParameter("keyword", keyword).getSingleResult();
When I use mysql on debian via SSH and type in SELECT query manually:
SELECT * FROM keywords WHERE keyword = 'ser żółty';
it also works and return single result. Encoding and collations of tables and columns are also ok. In datasource configuration I've added
?UseUnicode=true&characterEncoding=utf8
parameters, but it also did not help. I thought that maybe there is a problem with encoding in data from request send by form, but the problem appears even if the parameter i.e. "ser żółty" is hardcoded in my repository class.
I also use Hibernate Search for indexing and the FullTextEntityManager return correct results with Polish letters.
I assume that there is some problem between Hibernate and MySQL, but I have no more ideas what could I change. Any suggestions?
Server Wildfly9.0.1, MySQL 5.6
Ok the problem was in encoding on the mysql server level. It was set to latin1 by default. To fix this follow this question Change MySQL default character set to UTF-8 in my.cnf? and edit your my.cnf file.
I have an issue with Entity Framework 5.0. I'm working with Silverlight 5 and MySQL 5.6 too.
I need to set an environment MySQL variable before each connexion to the MySQL server.
E.g
SET #my_var = 'test';
Under Mysql I don't have any issues.
The following raises an EntityFrameworkException (syntax error near '#').
this.ObjectContext.CreateQuery<object>(" SET #my_var = 'test' ");
OR
this.ObjectContext.CreateQuery<object>(" CALL set_my_var('test') ");
This last method raises a MySQLException saying that a DataReader is already open and need to be closed.
this.ObjectContext.ExecuteStoreQuery<object>(" CALL set_my_var('test') ", null);
I also tried to set a MySQL system environment (no '#') with the same result every time.
Any help will be much appreciated !
Thank you.
I tried so many things that I misspelled my variable in my code.
So the following finaly worked : ctx.ExecuteStoreCommand("SET #my_var = 'test'");
I decided to leave the instruction in the method Initialize of my domain service. This method is inherited of the LinqToEntitiesDomainService class.
But you need to set Allow User Variables=True in your MySQL connection string
(ref : Is it possible to use a MySql User Defined Variable in a .NET MySqlCommand?)
You simply need to use a recent version of the MySQL Connector because older versions use the '#' mark to define SQL parameters so it could conflict with custom variables. Now it uses the '?' mark : http://dev.mysql.com/doc/refman/5.0/es/connector-net-examples-mysqlcommand.html
My library was already up to date (6.6.5).
Thank you for the help !
Since your statement is not a query (i.e. does not return any result) you should use ExecuteStoreCommand. Something like this should work:
ctx.ExecuteStoreCommand("SET #my_var = 'test'")
I have an old application that started failing after an upgrade from MySQL 5.0 to 5.1.
A bit of research indicated this is due to "strict mode" which prevents inserting certain types of "invalid" values which previously were just automatically converted to something reasonable.
I tried SET ##SESSION.sql_mode = '' and SET ##GLOBAL.sql_mode = '' but I still get the error.
Also tried commenting out sql_mode in the my.ini.
Is there a stronger, sort of "nuclear" option to fix this?
In my application I usually make sure that the MySQL connection is using traditional mode by issuing
SET SESSION sql_mode = 'ANSI_QUOTES,TRADITIONAL'
on each new connection. I presume that if you just issue
SET SESSION sql_mode = ''
on each new connection, you will have solved the problem.
You should be able to change the default SQL mode for new connections by issuing
SET GLOBAL sql_mode = ''
but you must use an account with sufficient privileges to do this or it won't work.
I think that if you want to make sure a particular SQL mode is in operation for your application, the most robust way to do so is to set it for each and every new connection.
To allow invalid dates, you need:
SET sql_mode = 'ALLOW_INVALID_DATES';
But you'd better fix your application.
I've connected to a MySQL database using Perl DBI. I would like to find out which database I'm connected to.
I don't think I can use:
$dbh->{Name}
because I call USE new_database and $dbh->{Name} only reports the database that I initially connected to.
Is there any trick or do I need to keep track of the database name?
Try just executing the query
select DATABASE();
From what I could find, the DBH has access to the DSN that you initially connected with, but not after you made the change. (There's probably a better way to switch databases.)
$dbh->{Name} returns the db name from your db handle.
If you connected to another db after connected with your dbh, using mysql query "USE db_name", and you did not setup a new perl DBI db handle, of course, $dbh->{Name} will return the first you previously connected to... It's not spontaneic generation.
So to get the connected db name once the db handle is set up - for DBI mysql:
sub get_dbname {
my ($dbh) = #_;
my $connected_db = $dbh->{name};
$connected_db =~ s/^dbname=([^;].*);host.*$/$1/;
return $connected_db;
}
You can ask mysql:
($dbname) = (each %{$dbh->selectrow_hashref("show tables")}) =~ /^Tables_in_(.*)/;
Update: obviously select DATABASE() is a better way to do it :)
When you create a connection object it is for a certain database. In DBI's case anyway. I I don't believe doing the SQL USE database_name will affect your connection instance at all. Maybe there is a select_db (My DBI is rusty) function for the connection object or you'll have to create a new connection to the new database for the connection instance to properly report it.
FWIW - probably not much - DBD::Informix keeps track of the current database, which can change if you do operations such as CREATE DATABASE. The $dbh->{Name} attribute is specified by the DBI spec as the name used when the handle is established. Consequently, there is an Informix-specific attribute $dbh->{ix_DatabaseName} that provides the actual current database name. See: perldoc DBD::Informix.
You could consider requesting the maintainer(s) of DBD::MySQL add a similar attribute.