MySQL changes UTF-8 to ASCII-8BIT - mysql

I've this scenario.
A movie title:
$ title = "La leyenda de Osaín"
With this encoding:
$ title.encoding.name
>> UTF-8
I then saves it to the database.
$ movie = Movie.create!(:title => title)
Then I try to get the movie.
$ Movie.find(movie.id).title.encoding.name
>> "ASCII-8BIT"
$ Movie.find(movie.id).title
>> "La leyenda de Osa\xC3\xADn"
All other movies works that does not contain special characters like í and û.
This is my database.yaml file:
development:
adapter: mysql
database: development
username: linus
password: my_password
socket: /tmp/mysql.sock
encoding: UTF8
I'm getting the right sort of data when using forced_encoding.
$ Movie.find(movie.id).title.force_encoding("UTF-8")
>> "La leyenda de Osaín"
I'm using Rails 3.0.5.rc1 with MySQL 14.14.
Anyone knows what the problem may be?

I found a solution to my problem.
Now I'm using the newer mysql2 gem.
I replaced gem "mysql" with gem "mysql2" inside the Gemfile.
Then I changed the database adapter inside the database.yaml file.
From:
development:
adapter: mysql
database: development
username: linus
password: my_password
socket: /tmp/mysql.sock
encoding: UTF8
To:
development:
adapter: mysql2
database: development
username: linus
password: my_password
socket: /tmp/mysql.sock
encoding: UTF8
I think this was the deal breaker in my case:
Taken from Github MySQL2
[...]It also forces the use of UTF-8 [or binary] for the connection [and all strings in 1.9[...]

According to this link, rails scaffolding creates varchar(255) columns in mysql. The mysql documentation says the following about varchar(255):
For example, a VARCHAR(255) column can
hold a string with a maximum length of
255 characters. Assuming that the
column uses the latin1 character set
(one byte per character), the actual
storage required is the length of the
string (L), plus one byte to record
the length of the string.
My guess is that the column type in the database doesn't support characters that are represented by more than one byte. This link has more information about common pitfalls in rails when dealing with unicode strings and more specifically, it says you need to create your database as utf8 like so:
CREATE_DATABASE my_web_two_zero_development DEFAULT CHARSET utf8;

Related

Why I get `ActiveRecord::NoDatabaseError: Unknown database` error when trying to use numbers and underline in database name?

I am using ruby-2.2.4, Rails 4.2.5 and MySQL 5.7.16 with gem mysql2 in my Ruby on Rails application. I have created database with name 123_4 and set database name in /config/database.yml.
Why I am getting error ActiveRecord::NoDatabaseError: Unknown database '1234' when trying rake db:migrate?
If I try to run rake db:create database with name 1234 will be created.
If I use 123_abc4 for database name everything is fine.
my database.yml content:
production:
adapter: mysql2
database: 123_4
host: localhost
username: user
password: "pass"
encoding: utf8
Identifiers may begin with a digit but unless quoted may not consist solely of digits.
MySQL Schema Object Name
So you can use 123_abc4 because it contains letters.
If it only includes number, you will need to quote it: '123_4'

How to add custom mysql database in symfony

First of all hello everyone !
I'm on a Symfony 1.4 website, and I don't really have knowledge in this framework.
The site already uses 2 database connections, in databases.yml
all:
doctrine:
class: sfDoctrineDatabase
param:
dsn: mysql:host=localhost;dbname=my_db_name
username: my_username
password: ***
charset: utf8
encoding: utf8
attributes:
default_table_collate: utf8_general_ci
default_table_charset: utf8
use_native_enum: true
# Base de test
test:
doctrine:
class: sfDoctrineDatabase
param:
dsn: mysql:host=localhost;dbname=my_other_db_name
username: my_username
password: ***
charset: utf8
encoding: utf8
attributes:
default_table_collate: utf8_general_ci
default_table_charset: utf8
use_native_enum: true
It's working I think, well I never use base test but I guess it works.
I need to use another database, not in localhost this time but from another server.
I tried so many thing, that i learn from stackoverflow post.
I created a config folder into my module, where I put the code into a new databases.yml
all:
doctrine:
class: sfDoctrineDatabase
param:
dsn: mysql:host=mysql_host.bdb;dbname=mysql_db_name
username: mysql_username
password: ***
charset: utf8
encoding: utf8
attributes:
default_table_collate: utf8_general_ci
default_table_charset: utf8
use_native_enum: true
I don't know how to access to this database, seems to not working at all.
So I try directly into my template, seems logic because I need this database only into 1 template, in response of a form.
$bd_nom_serveur='mysql_host.bdb';
$bd_login='mysql_user';
$bd_mot_de_passe='***';
$bd_nom_bd='mysql_db_name';
$base = mysqli_connect($bd_nom_serveur, $bd_login, $bd_mot_de_passe, $bd_nom_bd);
Sending me warning
Warning: mysqli_connect(): (HY000/2005): Unknown MySQL server host 'mysql_host.bdb' (2) in /srv/d_/www/draft.site.com/apps/frontend/modules/FormContactSlot/templates/sendMailCatalog.php on line 14
I also tried
$dsn = 'mysql:dbname=mysql_db_name;host=mysql_host.bdb';
$user = 'mysql_user';
$password = '***';
$dbh = new PDO($dsn, $user, $password);
$conn = Doctrine_Manager::connection($dbh);
sendig me error
500 | Internal Server Error | PDOException SQLSTATE[HY000] [2005]
Unknown MySQL server host 'mysql_host.bdb' (2)
I try with and without .bdb but nothing change.
I don't know how to access to this database from symfony 1.4
Can anyone help me please !
keep in mind that I have very little knowledge of Symfony, be gentle please !
EDIT :
I also tried another trick.
I add the database after my principal database on config/databases.yml
all:
doctrine:
class: sfDoctrineDatabase
param:
dsn: mysql:host=localhost;dbname=my_db_name
username: my_username
password: ***
charset: utf8
encoding: utf8
attributes:
default_table_collate: utf8_general_ci
default_table_charset: utf8
use_native_enum: true
client:
class: sfDoctrineDatabase
param:
dsn: mysql:host=host.bdb;dbname=dbname
username: user
password: ***
charset: utf8
encoding: utf8
attributes:
default_table_collate: utf8_general_ci
default_table_charset: utf8
use_native_enum: true
And this time I got an error whenever I try to go on my website. I also do the command symfony doctrine:build-schema but this send me the same error unknown mysql server...
Please I need help, I have to access my mysql database from symfony, I don't have the choice.
I respond to myself because I find a solution :
There is no solution.
I'm on a shared OVH solution, and shared OVH databases are not remotely accessible. So I have to take a vps solution, where there is IP host for database.
Thanks to Marek for helping me finding the solution.
Here the query that works.
$bd_nom_serveur='IP host';
$bd_login='mysql_user';
$bd_mot_de_passe='***';
$bd_nom_bd='mysql_db_name';
$base = mysqli_connect($bd_nom_serveur, $bd_login, $bd_mot_de_passe, $bd_nom_bd);
PDO still not working.

App and Database UTF-8 encoded but still exception

I develop a Rails app linked with a Mysql db.
In rails console
irb(main):002:0> Rails.application.config.encoding
=> "utf-8"
My database
I use Scalingo for deployment (kind of Heroku), my database.yml looks:
"production"=>{"adapter"=>"mysql", "database"=>"****", "pool"=>5, "timeout"=>10000, "encoding"=>"utf8", "username"=>"****", "password"=>"<password>", "port"=>30324, "host"=>"****"}}
My app and my DB are well configured but on write of name: 'é' on db for example it returns:
=> #<Node id: 21, name: "\xC3\xA9", created_at: "2015-05-18 06:43:29", updated_at: "2015-05-18 06:43:29">
How can be this problem fixed?
SOLUTION
replace gem mysql with mysql2 and also the adapter to mysql2
First you need to set up database.yml by define utf-8 of encoding
development:
adapter: mysql
encoding: utf8
database: my_international_db
username: user
password: password
socket: /var/run/mysqld/mysqld.sock
Now also check your conf file of mysql
in /etc/mysql/my.conf
...
[client]
default-character-set=utf8
...
[mysqld]
default-character-set=utf8
character-set-server=utf8
default-collation=utf8_unicode_ci
...
If the above are there then no need to do anything. If you change anything on it then restart mysql server
sudo /etc/init.d/mysql restart
Please let me know
Mysql is returning you UTF-8 encoded data, but the mysql activerecord adapter is not correctly setting the encoding for the strings. ASCII-8BIT just means a byte strings.
I'm guessing you are using the "mysql" adapter. Try using the more modern "mysql2" adapter. To do this add activerecord-mysql2-adapter to you Gemfile, and change your database.yml file so that the new adapter is used:
"adapter"=>"mysql2"

Generate a model that uses pre-existing data from database

Im making an application with a pre-existing database, I set up the database.yml to use the database.
database.yml
development:
adapter: mysql2
encoding: utf8
# database: ttlem_demo_development
database: ttle
pool: 5
username: root
password:
socket: /tmp/mysql.sock
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
adapter: mysql2
encoding: utf8
database: ttlem_demo_test
pool: 5
username: root
password:
socket: /tmp/mysql.sock
production:
adapter: mysql2
encoding: utf8
database: ttlem_demo_production
pool: 5
username: root
password:
socket: /tmp/mysql.sock
I only want one table out of the database it is called account views, I try to generate a scaffold for this with all the correct fields but it tells me i need to migrate (when i render it in the browser), if i migrate i wont be able to use the existing data, is this correct? How can i make a model that will use the existing data and fields?
Thank you for all your help :)
Two things to try:...
#1 Run your scaffold without a migration so it doesn't think you're missing one.
rails g scaffold yourmodelname fieldone:fieldtype ... etc --no-migration
#2 If that doesn't work you can go the long way round but dumping and reloading with a valid schema version number
Add this db to yml to your gemfile:
gem 'yaml_db', github: 'jetthoughts/yaml_db', ref: 'fb4b6bd7e12de3cffa93e0a298a1e5253d7e92ba'
It works for either rails 3 or rails 4.
Do a schema dump of your current database so you'll get a schema.rb with a valid version number.
bundle exec rake db:schema:dump
Now that you have a valid schema dump your data.
bundle exec rake db:data:dump
Drop your database (you can do it manually using the mysql commands if you prefer or run rake db:drop)
Now recreate it with your schema file.
bundle exec rake db:schema:load
Now add back your data
bundle exec rake db:data:load
Start your server and assuming your correctly matched all your data fields in your model (so the proper strong parameters are set from your scaffold) you should be good.

activerecord/mysql saves my latin1 data as utf8

In my database config I have:
adapter: mysql
database: xxx
username: xxx
password: xxx
host: localhost
encoding: 'latin1'
My table definition has: DEFAULT CHARSET=latin1
I put this at the top of my file:
# encoding: iso-8859-1
And yet still when I save my record and load it again the latin1 data has been mysteriously converted to utf8. Can someone suggest something I'm missing?
What about the columns' encoding? You have to set the connection, the tables, the columns and the program encoding/charset to the same one to have your code work properly.