Rails4, MySql, mysql2 encoding issue - mysql

I have a project where Rails 4, MySql 5.1, gem mysql2 are used.
This stack worked fine until today. I've added two fields in a table (added corresponding migration) and if I save data to the table, the data in this field will look ugly, like -%D0%9D%D0%BE%D0%B4%D0%B0%D1%80%D0%B8
Here is Rails log:
This is the data from a request. All looks fine, 'name' and 'comment' are in Russian.
Parameters: {"{\"user_id\":\"1\",\"name\":\"АБВ\",\"email\":\"mike#outofcloud.ru\",\"answer\":\"5\",\"comment\":\"цувцув\"}"=>nil}
But in the SQL statement the 'name' parameter changed:
SQL (23.3ms) INSERT INTO "feedbacks" ("answer", "code", "comment", "created_at", "email", "name", "updated_at", "user_id") VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING "id" [["answer", 5], ["code", 1], ["comment", "цувцув"], ["created_at", "2015-05-13 08:54:22.047321"], ["email", "example#example.ru"], ["name", **"%D0%90%D0%91%D0%92"**], ["updated_at", "2015-05-13 08:54:22.047321"], ["user_id", "1"]]
'comment' field was added before my migrations, it works as expected. It look great in the DB and in the app.
'name' field have been added recently and its doesn't work as expected.
Can someone help with the issue?

The string that you're showing is the result of using URI.encode on "АБВ" - see:
[4] pry(main)> URI.encode "АБВ"
=> "%D0%90%D0%91%D0%92"
Rails doesn't do this by itself. You must have some code somewhere else (or, hopefully not, a gem) that's performing that transformation.
If you don't know where that is then please add the code from your controllers showing any before_action methods, or other transformations. Also the code from your model if you're doing anything before_save or any custom attribute setters, eg. def name=

Related

Problems Parsing a Json File in Ruby

I'm pretty new to programming ruby and using json packages, I think I'm facing a typical noob-error, but I just cant find my mistake for two days now.. I've started like this:
require 'discogs-wrapper'
require 'json'
aw = Discogs::Wrapper.new("my_application", user_token: "my_user_token")
inv = aw.get_user_inventory('my_user_name', :per_page => 1, :page => 1)
p=JSON.parse(inv)
What I recieve is this:
C:/Ruby24-x64/lib/ruby/2.4.0/json/common.rb:156:in `initialize': no implicit conversion of Hashie::Mash into String (TypeError)
from C:/Ruby24-x64/lib/ruby/2.4.0/json/common.rb:156:in `new'
from C:/Ruby24-x64/lib/ruby/2.4.0/json/common.rb:156:in `parse'
from C:/Users/rtuz2th/.RubyMine2017.3/config/scratches/scratch.rb:6:in `<top (required)>'
from -e:1:in `load'
from -e:1:in `<main>'
I've already been searching your forum and I've learned that this error probably means that I do not have a correct JSON-file (although it comes straight from the discogs-api, I basically thought these guys know what they do). So what I tried next was
require 'discogs-wrapper'
require 'json'
aw = Discogs::Wrapper.new("my_application", user_token: "my_user_token")
inv = aw.get_user_inventory('my_user_name', :per_page => 1, :page => 1)
inv = inv.to_json
p=JSON.parse(inv)
So at least, my code runs without any errors now. But when I try to refer to any information in inv, I just get empty responses. inv (without the .to_json looks like this:
{"pagination"=>
{"per_page"=>1,
"items"=>13692,
"page"=>1,
"urls"=>
{"last"=>
"URL containing tokens",
"next"=>
"URL containing tokens"},
"pages"=>13692},
"listings"=>
[{"status"=>"Draft",
"original_price"=>
{"curr_abbr"=>"EUR",
"formatted"=>"\u20AC10.46",
"value"=>10.46,
"curr_id"=>3},
"weight"=>230.0,
"original_shipping_price"=>
{"curr_abbr"=>"EUR",
"formatted"=>"\u20AC5.90",
"value"=>5.9,
"curr_id"=>3},
"price"=>{"currency"=>"EUR", "value"=>10.46},
"allow_offers"=>true,
"uri"=>"https://www.discogs.com/sell/item/437965910",
"sleeve_condition"=>"Very Good Plus (VG+)",
"format_quantity"=>1,
"id"=>437965910,
"shipping_price"=>{"currency"=>"EUR", "value"=>5.9},
"posted"=>"2017-02-07T23:43:01-08:00",
"ships_from"=>"Germany",
"in_cart"=>false,
"comments"=>"",
"seller"=>
{"username"=>"zweischeiben.de",
"stats"=>{"rating"=>"100.0", "total"=>143, "stars"=>5.0},
"uid"=>3359767,
"url"=>"https://api.discogs.com/users/zweischeiben.de",
"html_url"=>"https://www.discogs.com/user/zweischeiben.de",
"shipping"=>
"(Lots of information about shipping, cut just cut it out)"
"payment"=>"Bank Transfer, PayPal",
"avatar_url"=>
"(cut out url)",
"resource_url"=>"https://api.discogs.com/users/zweischeiben.de",
"id"=>3359767},
"condition"=>"Near Mint (NM or M-)",
"release"=>
{"thumbnail"=>
"(cut out url)",
"description"=>"Steamhammer - Mountains (LP, Album, RE)",
"artist"=>"Steamhammer",
"format"=>"LP, Album, RE",
"resource_url"=>"https://api.discogs.com/releases/7303333",
"title"=>"Mountains",
"year"=>0,
"id"=>7303333,
"catalog_number"=>"201.006, 0201.006"},
"resource_url"=>"https://api.discogs.com/marketplace/listings/437965910",
"audio"=>false,
"external_id"=>"3, 16",
"location"=>"T52574"}]}
But if I now try to refer to eg the location:
require 'discogs-wrapper'
require 'json'
aw = Discogs::Wrapper.new("my_application", user_token: "my_token")
inv=aw.get_user_inventory('my_username', :per_page => 1, :page => 1)
inv=inv.to_json
p=JSON.parse(inv)
puts p[location]
I'm just getting an error like this
C:\Ruby24-x64\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) C:/Users/rtuz2th/.RubyMine2017.3/config/scratches/scratch.rb
C:/Users/rtuz2th/.RubyMine2017.3/config/scratches/scratch.rb:9:in `<top (required)>': undefined local variable or method `location' for main:Object (NameError)
from -e:1:in `load'
from -e:1:in `<main>'
Process finished with exit code 1
or an empty response. I'm trying to solve this for two days now, but I'm absolutely out of ideas. Thanks for your help in advance!
inv = inv.to_json
p = JSON.parse(inv)
The above is a noop. Use inv as is, it’s a hash already.
puts p[location]
You cannot just lookup whatever key located deeply in the tree hierarchy or you hash. You hash has stings as keys, it has a two top-level elements, 'pagination' and 'listings', the latter is an array, and each element of this array is a hash having location key.
While you were cutting the long value, you dropped the ending comma, making me unable to copy-paste the input and provide the exact answer.

get ids as well for all models with seed_dump gem

Hi I am using seed_dump gem to create seeds.rb from existing data but I stuck one thing I want to get ids for all models as well how can I do this for example currently if I run
rake db:seed:dump
I just get code like this
Product.create(title: "title", description: "text")
but I want this
Product.create(id: 1, title: "title", description: "text")
how can i do that?
rake db:seed:dump EXCLUDE=[]
This overwrites the default exclude of [:id, :created_at, :updated_at] so that it includes the id
Create your own export. Assuming that your model's name is Country:
lib/tasks/export.rake
namespace :export do
desc "Exports data for using in a seeds.rb."
task :seeds_format => :environment do
Country.order(:id).all.each do |country|
puts "Country.create(#{country.serializable_hash.
delete_if {|key, value| ['created_at','updated_at'].
include?(key)}.to_s.gsub(/[{}]/,'')})"
end
end
end
You could run that with this command:
rake export:seeds_format > db/seeds.rb

Ruby/ActiveRecord not detecting changes after stripping whitespace from field

I am trying to remove whitespace and carriage returns from various fields in a MySQL database using the mysql2 adapter and ActiveRecord:
Ruby 1.9.3p194
ActiveRecord 3.2.8
MySQL 5.5.28
foo = People.find(1)
foo.name => "\rJohn Jones"
foo.name.lstrip! => "John Jones"
foo.name => "John Jones"
foo.changes => {} #no changes detected to foo.name???
foo.save => true # but does nothing to database.
if I do:
foo.name = "John Jones"
foo.save => true
People.find(1).name => "John Jones" # this works and saves to database
I've searched all over for this... any suggestions?
When you do in-place modifications on model attributes, no assignment occurs and the model is unaware any changes have been made. The correct way to do this is to reassign:
foo.name = foo.name.lstrip
This triggers the name= method and the dirty tracking is engaged.

Rails 3.1 and Mysql Position in table

Using Rails 3.1 and mysql,I've the following issue: I've created the array #addresses:
#registry_all = Registry.all
#addresses = #registry_all.map{|reg| [ reg.id, reg.indirizzo_1, reg.indirizzo_2, reg.indirizzo_3, reg.indirizzo_4 ]}.sort
and this is model relative to Registry:
rails g model registry id:integer, created_at:datetime, updated_at:datetime, name:string, cod_fiscale:string :limit=>16, p_iva:string :limit=>11, indirizzo_1:text, indirizzo_2:text, indirizzo_3:text, indirizzo_4:text ,telefono1:string :limit=>25, telefono2:string :limit=>25
So array #addresses contains all registry's addresses and now I want to retrieve all addresses owned by a singular registry, making a finding by registry.id. I mean,I was thinking something similar to that:
#addresses.find[registry.id]
but off course doing so, (supposing that registry.id has equals to 30), I retrieve all addresses owned by 30-th element of array #addresses and not addresses owned by registry.id. This should works only if object for what I'm looking for, is 30-th element of array addresses, in other words, only if this object is 30-th inside Registry mysql table
How can I do this?
If I understood well you have all the 4 addresses [1] for a registry in a row of the registries DB table with the names indirizzo_1, indirizzo_2, indirizzo_3, indirizzo_4 and you want these addresses in an array. You could do this:
registry = Registry.find(the_registry_id)
return [registry.indirizzo_1, registry.indirizzo_2, registry.indirizzo_3, registry.indirizzo_4]
With find you get the Registry with that ID and then you use return the 4 attributes as an array.
But from a design point of view I would not add the 4 fields for the addresses but I would use a one-to-many relationship since a registry can have many addresses.
[1] "indirizzo" is "address" in italian
I created a migration somewhat like yours (minus the :limit)
class CreateRegistries < ActiveRecord::Migration
def change
create_table :registries do |t|
t.integer :id
t.datetime :created_at
t.datetime :updated_at
t.string :name
t.string :cod_fiscale
t.string :p_iva
t.text :indirizzo_1
t.text :indirizzo_2
t.text :indirizzo_3
t.text :indirizzo_4
t.string :telefono1
t.string :telefono2
t.timestamps
end
end
end
I assume you have already done:
$ bundle exec rake db:migrate
A simple way to think of this is to open a rails console:
rails c
I created 5 Registry records like this:
Registry.create(name:"name5", cod_fiscale:"cod_fiscale", p_iva:"p_iva", indirizzo_1:"txt", indirizzo_2:"txt", indirizzo_3:"txt", indirizzo_4:"txt" ,telefono1:"12345")
So now typing:
Registry.count
Gives me:
1.9.2-p180 :017 > Registry.count
(0.2ms) SELECT COUNT(*) FROM "registries"
=> 6
You can now use the .find method like this:
Registry.find(1)
where (1) is the id of one of the 6 records we have.
If using this in a Controller it will look like this:
#registry = Registry.find(params[:id])
Then you could grab the indirizzo by typing:
Registry.find(1).indirizzo_1
which gives:
1.9.2-p180 :036 > Registry.find(1).indirizzo_1
Registry Load (0.3ms) SELECT "registries".* FROM "registries" WHERE "registries"."id" = ? LIMIT 1 [["id", 1]]
=> "txt"
Let me know if you need more help and feel free to include more output.

Ruby On Rails: Testing deletes tables

I'm creating an application in RoR and I'm implementing unit testing in all my models.
When I run every test on his own (by running ruby test/unit/some_test.rb) all tests are successful.
But when I run all tests together (by running rake test:units) some tables from both databases (development e test) are deleted.
I'm using raw SQL (mysql) do create tables because I need composite primary keys and physical constraints so I figured it would be the best. Maybe this be the cause?
All my tests are in this form:
require File.dirname(FILE) + '/../test_helper'
require File.dirname(FILE) + '/../../app/models/order'
class OrderTestCase < Test::Unit::TestCase
def setup
#order = Order.new(
:user_id => 1,
:total => 10.23,
:date => Date.today,
:status => 'processing',
:date_concluded => Date.today,
:user_address_user_id => 3,
:user_address_address_id => 5,
:creation_date => Date.today,
:update_date => Date.today
)
end
################ Happy Path
def test_happy_path
assert #order.valid?, #order.errors.full_messages
end
...
The errors I get when running the tests are something like this:
3) Error:
test_empty_is_primary(AddressTestCase):
ActiveRecord::StatementInvalid: Mysql::Error: Table 'shopshop_enterprise_test.addresses' doesn't exist: SHOW FIELDS FROM addresses
/test/unit/address_test.rb:9:in new'
/test/unit/address_test.rb:9:insetup'
Any guesses?
Thanks!
PS: When using postgres as the database engine, everything works fine with rake test:units! (of course, with the correct changes so the sql statements can work with postgres)