Symbolize keys in MySQL JSON fields using ActiveRecord - mysql

I have a JSON column in one of my models, that holds an array of hashes
create_table "articles", options: "ENGINE=InnoDB DEFAULT CHARSET=utf8", force: :cascade do |t|
t.string "title"
t.json "data"
end
When I store a Ruby hash inside #data, ActiveRecords takes care of converting it to json and storing it in the DB.
article = Article.new(data: [{ some: "data" }])
article.save
article.reload.data
=> [{ "some" => "data" }]
But as you can see in the example above, when I read the field back from the DB, the keys are converted to strings (this is obvious because it's stored as JSON).
My question is: Does ActiveRecord provide any built-in functionality to convert the string keys to symbols?
I could overwrite the getter and symbolize the keys inplace, but this becomes unhandy if you have a lot of json fields:
def data
_data = self[:data]
_data.each.with_index do |hash, idx|
_data[idx] = hash.symbolize_keys
end
_data
end
It becomes more messy when you have mixed data types inside the same array and you have to take care if it is a hash or not...
# this would fail because String does not implement #symbolize_keys
article.data << "foo"
article.data << { bar: "test" }

There is a way to convert the string keys to symbols, if it is a valid hash.
JSON.parse(article.reload.data,:symbolize_names => true)
This method will symbolize all keys recursively. It wont preserve any mixing of symbol and string.
But in your case if you have missed data types you also need to check whether the element of the array is a hash or string. You might try this using Kernel#eval :
def valid_hash?(string)
eval(string).is_a?(Hash)
rescue SyntaxError
false
end
More details for eval method is here

Related

How to serialize/deserialize ruby hashes/structs with objects as keys to json

I would like to dump a nested datastructure in ruby to json (I am aware of the Marshal module but I need a standard format) and be able to load/parse the datastructure again. Catch: I use structs (or easier for the example: hashes) as keys of hashes. Example:
require 'json'
h = {{hello: 123} => 123}
JSON.parse(JSON.generate(h)) #=> {"{:hello=>123}"=>123}
So the problem is, that JSON.generate(h) serialises the key {:hello=>123} as a string and when I parse the result again, it remains a string.
How can I solve this and regain the original structure after generate/parse?
JSON only allows strings as object keys. For this reason to_s is called for all keys.
You'll have the following options to solve your issue:
The best option is changing the data structure so it can properly be serialized to JSON.
You'll have to handle the stringified key yourself. An Hash produces a perfectly valid Ruby syntax when converted to a string that can be converted using Kernel#eval like Andrey Deineko suggested in the comments.
result = json.transform_keys { |key| eval(key) }
# json.transform_keys(&method(:eval)) is the same as the above.
The Hash#transform_keys method is relatively new (available since Ruby 2.5.0) and might currently not be in you development environment. You can replace this with a simple Enumerable#map if needed.
result = json.map { |k, v| [eval(k), v] }.to_h
Note: If the incoming JSON contains any user generated content I highly sugest you stay away from using eval since you might allow the user to execute code on your server.
I need a standard format
YAML is a standard format that would suffice here:
▶ h = {{hello: 123} => 123}
#⇒ {{:hello=>123}=>123}
▶ YAML.dump h
#⇒ "---\n? :hello: 123\n: 123\n"
▶ YAML.load _
#⇒ {{:hello=>123}=>123}
As already pointed by mudasobwa, YAML is a good tool: allows you to store also custom class objects:
require 'yaml'
class MyCaptain
attr_accessor :name, :ship
def initialize(name, ship)
#name = name
#ship = ship
end
end
kirk = MyCaptain.new('James T. Kirk', 'USS Enterprise NCC-1701')
picard = MyCaptain.new('Jean-Luc Picard', 'Enterprise NCC-1701D')
captains = [kirk, picard]
File.open("my_captains.yml","w") do |file|
file.write captains.to_yaml
end
p YAML.load_file('my_captains.yml')
#=> [#<MyCaptain:0x007f889d0973b0 #name="James T. Kirk", #ship="USS Enterprise NCC-1701">, #<MyCaptain:0x007f889d096b40 #name="Jean-Luc Picard", #ship="Enterprise NCC-1701D">]

Crystal handle json file of known format but dynamic keys

So I have a JSON file of a somewhat known format { String => JSON::Type, ... }. So it is basically of type Hash(String, JSON::Type). But when I try and read it from file to memory like so: JSON.parse(File.read(#cache_file)).as(Hash(String, JSON::Type)) I always get an exception: can't cast JSON::Any to Hash(String, JSON::Type)
I'm not sure how I am supposed to handle the data if I can't cast it.
What I basically want to do is the following:
save JSON::Type data under a String key
replace JSON::Type data with other JSON::Type data under a String key
And of course read from / write to file...
Here's the whole thing I've got so far:
class Cache
def initialize(#cache_file = "/tmp/cache_file.tmp")
end
def cache(cache_key : (String | Symbol))
mutable_cache_data = data
value = mutable_cache_data[cache_key.to_s] ||= yield.as(JSON::Type)
File.write #cache_file, mutable_cache_data
value
end
def clear
File.delete #cache_file
end
def data
unless File.exists? #cache_file
File.write #cache_file, {} of String => JSON::Type
end
JSON.parse(File.read(#cache_file)).as(Hash(String, JSON::Type))
end
end
puts Cache.new.cache(:something) { 10 } # => 10
puts Cache.new.cache(:something) { 'a' } # => 10
TL;DR I want to read a JSON file into a Hash(String => i_dont_care), replace a value under a given key name and serialize it to file again. How do I do that?
JSON.parse returns an JSON::Any, not a Hash so you can't cast it. You can however access the underlying raw value as JSON.parse(file).raw and cast this as hash.
Then your code is basically working (I've fixed a few error): https://carc.in/#/r/28c1
You can use use Hash(String, JSON::Type).from_json(File.read(#cache_file)). Hopefully you can restrict the type of JSON::Type down to something more sensible too. JSON::Any and JSON.parse_raw are very much a last resort compared to simply representing your schema using Hash, Array and custom types using JSON.mapping.

How to use ransack to search MySQL JSON array in Rails 5

Rails 5 now support native JSON data type in MySQL, so if I have a column data that contains an array: ["a", "b", "c"], and I want to search if this column contains values, so basically I would like to have something like: data_json_cont: ["b"]. So can this query be built using ransack ?
Well I found quite some way to do this with Arrays(not sure about json contains for hash in mysq). First include this code in your active record model:
self.columns.select{|column| column.type == :json}.each do |column|
ransacker "#{column.name}_json_contains".to_sym,
args: [:parent, :ransacker_args] do |parent, args|
query_parts = args.map do |val|
"JSON_CONTAINS(#{column.name}, '#{val.to_json}')"
end
query = query_parts.join(" * ")
Arel.sql(query)
end
end
Then assuming you have class Shirt with column size, then you can do the following:
search = Shirt.ransack(
c: [{
a: {
'0' => {
name: 'size_json_contains',
ransacker_args: ["L", "XL"]
}
},
p: 'eq',
v: [1]
}]
)
search.result
It works as follows: It checks that the array stored in the json column contains all elements of the asked array, by getting the result of each json contains alone, then multiplying them all, and comparing them to arel predicate eq with 1 :) You can do the same with OR, by using bitwise OR instead of multiplication.

ruby/hash:ActiveSupport::HashWithIndifferentAccess

I store a hash to the mysql,but the result make me confuse:
hash:
{:unique_id=>35, :description=>nil, :title=>{"all"=>"test", "en"=>"test"}...}
and I use the serialize in my model.
serialize :title
The result in mysql like this:
--- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
all: test
en: test
Anyone can tell me what is the meaning? Why there is a ruby/hash:ActiveSupport::HashWithIndifferentAccess in mysql?
TL;DR:
serialize :title, Hash
What’s happening here is that serialize internally will yaml-dump the class instance. And the hashes in rails are monkeypatched to may having an indifferent access. The latter means, that you are free to use both strings and respective symbols as it’s keys:
h = { 'a' => 42 }.with_indifferent_access
puts h[:a]
#⇒ 42
Hash needs to be serialized, default serializer is YAML which supports in some way, in the Ruby implementation, the storing of type. Your hash is of type ActiveSupport::HashWithIndifferentAccess, so when fetched back, ruby knows what object should get back (unserializes it to an HashWithIndifferentAccess).
Notice that HashWithIndifferentAccess is a hash where you can access values by using either strings or symbols, so:
tmp = { foo: 'bar' }.with_indifferent_access
tmp[:foo] # => 'bar'
tmp['foo'] # => 'bar'
With a normal hash (Hash.new), you would get instead:
tmp = { foo: 'bar' }
tmp[:foo] # => 'bar'
tmp['foo'] # => nil
Obviously, in mysql, the hash is stored as a simple string, (YAML is plain text with a convention)

How can I get ruby's JSON to follow object references like Pry/PP?

I've stared at this so long I'm going in circles...
I'm using the rbvmomi gem, and in Pry, when I display an object, it recurses down thru the structure showing me the nested objects - but to_json seems to "dig down" into some objects, but just dump the reference for others> Here's an example:
[24] pry(main)> g
=> [GuestNicInfo(
connected: true,
deviceConfigId: 4000,
dynamicProperty: [],
ipAddress: ["10.102.155.146"],
ipConfig: NetIpConfigInfo(
dynamicProperty: [],
ipAddress: [NetIpConfigInfoIpAddress(
dynamicProperty: [],
ipAddress: "10.102.155.146",
prefixLength: 20,
state: "preferred"
)]
),
macAddress: "00:50:56:a0:56:9d",
network: "F5_Real_VM_IPs"
)]
[25] pry(main)> g.to_json
=> "[\"#<RbVmomi::VIM::GuestNicInfo:0x000000085ecc68>\"]"
Pry apparently just uses a souped-up pp, and while "pp g" gives me close to what I want, I'm kinda steering as hard as I can toward json so that I don't need a custom parser to load up and manipulate the results.
The question is - how can I get the json module to dig down like pp does? And if the answer is "you can't" - any other suggestions for achieving the goal? I'm not married to json - if I can get the data serialized and read it back later (without writing something to parse pp output... which may already exist and I should look for it), then it's all win.
My "real" goal here is to slurp up a bunch of info from our vsphere stuff via rbvmomi so that I can do some network/vm analysis on it, which is why I'd like to get it in a nice machine-parsed format. If I'm doing something stupid here and there's an easier way to go about this - lay it on me, I'm not proud. Thank you all for your time and attention.
Update: Based on Arnie's response, I added this monkeypatch to my script:
class RbVmomi::BasicTypes::DataObject
def to_json(*args)
h = self.props
m = h.merge({ JSON.create_id => self.class.name })
m.to_json(*args)
end
end
and now my to_json recurses down nicely. I'll see about submitting this (or the def, really) to the project.
The .to_json works in a recursive manner, the default behavior is defined as:
Converts this object to a string (calling to_s), converts it to a JSON string, and returns the result. This is a fallback, if no special method to_json was defined for some object.
json library has added some implementation for some common classes (check the left hand side of this documentation), such as Array, Range, DateTime.
For an array, to_json first convert all the elements to json object, concat then together, and then add the array mark [/].
For your case, you need to define your customized to_json method for GuestNicInfo, NetIpConfigInfo and NetIpConfigInfoIpAddress. I don't know your implementation about these three classes, so I wrote a example to demonstrate how to achieve this:
require 'json'
class MyClass
attr_accessor :a, :b
def initialize(a, b)
#a = a
#b = b
end
end
data = [MyClass.new(1, "foobar")]
puts data.to_json
#=> ["#<MyClass:0x007fb6626c7260>"]
class MyClass
def to_json(*args)
{
JSON.create_id => self.class.name,
:a => a,
:b => b
}.to_json(*args)
end
end
puts data.to_json
#=> [{"json_class":"MyClass","a":1,"b":"foobar"}]