Ruby using a date as key in a hash from MySQL - mysql

I have a hash which is the result of a .map method on a MySQL2::Result object which looks like this:
{#<Date: 2018-01-02 ((2458121j,0s,0n),+0s,2299161j)>=>"OL,BD,DM,WW,DG"}
{#<Date: 2018-01-03 ((2458122j,0s,0n),+0s,2299161j)>=>"KP,LW"}
{#<Date: 2018-01-04 ((2458123j,0s,0n),+0s,2299161j)>=>"LW,WW,FS,DG"}
{#<Date: 2018-01-05 ((2458124j,0s,0n),+0s,2299161j)>=>"OL,KP,BD,SB,LW,DM,AS,WW,FS,DG"}
{#<Date: 2018-01-06 ((2458125j,0s,0n),+0s,2299161j)>=>"OL,KP,BD,SB,LW,DM,AS,WW,FS,DG"}
I would like to pull the values (the two letter items) from the hash, by referencing with the key.
I have tried
puts hash_name["2018-01-06"]
puts hash_name['2018-01-06']
puts hash_name[Date.new(2018,1,6)]
puts hash_name["<Date: 2018-01-06 ((2458125j,0s,0n),+0s,2299161j)>"]
puts hash_name["#<Date: 2018-01-06 ((2458125j,0s,0n),+0s,2299161j)>"]
All return nothing or an error.
The hash is created by doing the following:
hash_name = #available_items.map do
|h| {h["tdate"] => h["items"] }
end
Is there something I can do during the creation of the hash, or now, to be able to easily pull the value out using e.g. can I convert it to some other date format like ISO format?
Thanks

I think your problem is that Enumerable#map doesn't do what you think it does. This:
hash_name = #available_items.map do
|h| {h["tdate"] => h["items"] }
end
will give you an array of single entry hashes, the individual hashes will map Dates to strings but the result looks like:
[
{ date1 => string1 },
{ date2 => string2 },
...
]
rather than:
{
date1 => string1,
date2 => string2,
...
}
as you're expecting. Switching to #each_with_object should take care of your problem:
hash_name = #available_items.each_with_object({}) do |row, h|
h[row['tdate']] = row['items']
end

You're close here, but you're generating an array of hashes, not a singular hash:
hash_name = #available_items.map do |i|
[ i["tdate"], i["items"] ]
end.to_h
This creates an array of key/value pair arrays, then converts them to a hash with the .to_h method.
You can also use group_by if your input data can be grouped neatly, like:
hash_name = #available_items.group_by do |i|
i['tdate']
end
Where that approach might be good enough if can deal with the output format. It's keyed by date.
Note that using symbol keys like :tdate and :items is usually preferable to string keys. It's worth trying to steer towards that in most cases where there'd otherwise be rampant repetition of those strings.

In the hopes that this may help others to do a similar thing, here is what I ended up doing.
I have a MySQL2::Result object as shown above, on which I run:
#available_hash = #available_items.map do |row|
[ row["tdate"], row["available"] ]
end.to_h
Having previously declared a start_date and an end_date I then select an available item from the list, at random to fill a new hash using the dates as keys:
$final_hash = Hash.new("")
for date in (start_date..end_date)
#available_today = #available_hash[date].to_s.split(",")
$final_hash[date] = random_item(#available_today)
date +=1;
end
Whilst I am sure there is probably a more elegant way of doing this, I am delighted that you have helped me to get this to work!

Obviously hash map is not suitable for a date as the key, hash map is more suitable for key as id, tag, etc. It should be a unique key.
Please provide more information about what you need to do with this hash map, for sure you can have some more clever data structure.
If you have an array with two keys (tdate, items) and you want to lookup for the date just use select:
result = available_items.select { |elem| elem['tdate'] === Date.new(2001,2,3) }
reference for '===' operator in Date class
http://ruby-doc.org/stdlib-2.1.1/libdoc/date/rdoc/Date.html#method-i-3D-3D-3D

Related

Pass data from JSON to variable for comparison

I have a request that I make in an API using GET
LWP::UserAgent,
the data is returned as JSON, with up to two results at most as follows:
{
"status":1,
"time":1507891855,
"response":{
"prices":{
"nome1\u2122":{
"preco1":1111,
"preco2":1585,
"preco3":1099
},
"nome2":{
"preco1":519,
"preco2":731,
"preco3":491
}
}
}
}
Dump:
$VAR1 = {
'status' => 1,
'time' => 1507891855,
'response' => {
'prices' => {
'nome1' => {
'preco1' => 1111,
'preco3' => 1099,
'preco2' => 1585
},
'nome2' => {
'preco3' => 491,
'preco1' => 519,
'preco2' => 731
}
}
}
};
What I would like to do is:
Take this data and save it in a variable to make a comparison using if with another variable that already has the name stored. The comparison would be with name1 / name2 and if it is true with the other variable it would get preco2 and preco3 to print everything
My biggest problem in the case is that some of these names in JSON contain characters like (TradeMark) that comes as \u2122 (some cases are other characters), so I can not make the comparison with the name of the other variable that is already with the correct name
nome1™
If I could only save the JSON already "converted" the characters would help me with the rest.
Basically after doing the request for the API I want to save the contents in a variable already converting all \u2122 to their respective character (this is the part that I do not know how to do in Perl) and then using another variable to compare them names are equal to show the price
Thanks for the help and any questions please tell me that I try to explain again in another way.
If I understand correctly, you need to get the JSON that you receive in UTF8 format to an internal variable that you can process. For that, you may use JSON::XS:
use utf8;
use JSON::XS;
my $name = "nome1™";
my $var1 = decode_json $utf8_encoded_json_text;
# Compare with name in $name
if( defined $var1->{'response'}->{'prices'}->{$name} ) {
# Do something with the name that matches
my $match = $var1->{'response'}->{'prices'}->{$name};
print $match->{'preco1'}, "\n";
}
Make sure you tell the Perl interpreter that your source is in UTF8 by specifying use utf8; at the beginning of the script. Then make sure you are editing the script with an editor that supports that format.
The function decode_json will return a ref to the converted value. In this case a hash ref. From there you work your way into the JSON.
If you know $name is going to be in the JSON you may omit the defined part. Otherwise, the defined clause will tell you whether the hash value is there. One you know, you may do something with it. If the hash values are a single word with no special characters, you may use $var1->{response}->{prices}->{$name}, but it is always safer to use $var1->{'response'}->{'prices'}->{$name}. Perl gets a bit ugly handling hash refs...
By the way, in JSON::XS you will also find the encode_json function to do the opposite and also an object oriented interface.

Crystal handle json file of known format but dynamic keys

So I have a JSON file of a somewhat known format { String => JSON::Type, ... }. So it is basically of type Hash(String, JSON::Type). But when I try and read it from file to memory like so: JSON.parse(File.read(#cache_file)).as(Hash(String, JSON::Type)) I always get an exception: can't cast JSON::Any to Hash(String, JSON::Type)
I'm not sure how I am supposed to handle the data if I can't cast it.
What I basically want to do is the following:
save JSON::Type data under a String key
replace JSON::Type data with other JSON::Type data under a String key
And of course read from / write to file...
Here's the whole thing I've got so far:
class Cache
def initialize(#cache_file = "/tmp/cache_file.tmp")
end
def cache(cache_key : (String | Symbol))
mutable_cache_data = data
value = mutable_cache_data[cache_key.to_s] ||= yield.as(JSON::Type)
File.write #cache_file, mutable_cache_data
value
end
def clear
File.delete #cache_file
end
def data
unless File.exists? #cache_file
File.write #cache_file, {} of String => JSON::Type
end
JSON.parse(File.read(#cache_file)).as(Hash(String, JSON::Type))
end
end
puts Cache.new.cache(:something) { 10 } # => 10
puts Cache.new.cache(:something) { 'a' } # => 10
TL;DR I want to read a JSON file into a Hash(String => i_dont_care), replace a value under a given key name and serialize it to file again. How do I do that?
JSON.parse returns an JSON::Any, not a Hash so you can't cast it. You can however access the underlying raw value as JSON.parse(file).raw and cast this as hash.
Then your code is basically working (I've fixed a few error): https://carc.in/#/r/28c1
You can use use Hash(String, JSON::Type).from_json(File.read(#cache_file)). Hopefully you can restrict the type of JSON::Type down to something more sensible too. JSON::Any and JSON.parse_raw are very much a last resort compared to simply representing your schema using Hash, Array and custom types using JSON.mapping.

How to use ransack to search MySQL JSON array in Rails 5

Rails 5 now support native JSON data type in MySQL, so if I have a column data that contains an array: ["a", "b", "c"], and I want to search if this column contains values, so basically I would like to have something like: data_json_cont: ["b"]. So can this query be built using ransack ?
Well I found quite some way to do this with Arrays(not sure about json contains for hash in mysq). First include this code in your active record model:
self.columns.select{|column| column.type == :json}.each do |column|
ransacker "#{column.name}_json_contains".to_sym,
args: [:parent, :ransacker_args] do |parent, args|
query_parts = args.map do |val|
"JSON_CONTAINS(#{column.name}, '#{val.to_json}')"
end
query = query_parts.join(" * ")
Arel.sql(query)
end
end
Then assuming you have class Shirt with column size, then you can do the following:
search = Shirt.ransack(
c: [{
a: {
'0' => {
name: 'size_json_contains',
ransacker_args: ["L", "XL"]
}
},
p: 'eq',
v: [1]
}]
)
search.result
It works as follows: It checks that the array stored in the json column contains all elements of the asked array, by getting the result of each json contains alone, then multiplying them all, and comparing them to arel predicate eq with 1 :) You can do the same with OR, by using bitwise OR instead of multiplication.

Logstash - Substring from CSV column

I want to import many informations from a CSV file to Elastic Search.
My issue is I don't how can I use a equivalent of substring to select information into a CSV column.
In my case I have a field date (YYYYMMDD) and I want to have (YYYY-MM-DD).
I use filter, mutate, gsub like:
filter
{
mutate
{
gsub => ["date", "[0123456789][0123456789][0123456789][0123456789][0123456789][0123456789][0123456789][0123456789]", "[0123456789][0123456789][0123456789][0123456789]-[0123456789][0123456789]-[0123456789][0123456789]"]
}
}
But my result is false.
I can indentified my string but I don't how can I extract part of this.
My target it's to have something like:
gsub => ["date", "[0123456789][0123456789][0123456789][0123456789][0123456789][0123456789][0123456789][0123456789]","%{date}(0..3}-%{date}(4..5)-%{date}"(6..7)]
%{date}(0..3} : select from the first to the 4 characters of csv columns date
You can use ruby plugin to do conversion. As you say, you will have a date field. So, we can use it directly in ruby
filter {
ruby {
code => "
date = Time.strptime(event['date'],'%Y%m%d')
event['date_new'] = date.strftime('%Y-%m-%d')
"
}
}
The date_new field is the format you want.
First, you can use a regexp range to match a sequence, so rather than [0123456789], you can do [0-9]. If you know there will be 4 numbers, you can do [0-9]{4}.
Second, you want to "capture" parts of your input string and reorder them in the output. For that, you need capture groups:
([0-9]{4})([0-9]{2})([0-9]{2})
where parens define the groups. Then you can reference those on the right side of your gsub:
\1-\2-\3
\1 is the first capture group, etc.
You might also consider getting these three fields when you do the grok{}, and then putting them together again later (perhaps with add_field).

Couchbase Multiple Keys

I presume a simple question. I have the following data.
I want to search for all rows where the ID is > 2 but < 8 and the Price is > 30
I have used various versions of: startkey=["2", null] or even something like startkey=["2", "30"] just for testing.
It only ever seems to run both conditions on the first row. So if I do: startkey=["2", "30"] then I get back:
{"id":"3","key":["3","30"],"value":null},
{"id":"4","key":["4","30"],"value":null},
{"id":"5","key":["5","20"],"value":null},
{"id":"6","key":["6","60"],"value":null},
{"id":"8","key":["8","60"],"value":null}
Why is row 5 there?
I am starting to get the view that I need to handle this in the code (.net) and make multiple calls somehow... I can't seem to find anything on this that works....
Note: I have tried doing say a loop with for (i = 0; i < doc.ID.length; i++) and then using doc.ID[i] but it never returns anything....
Currently I just have
function (doc, meta) {
emit([doc.ID, doc.Price ],null);
}
Essentially I want to have a search where there are 5 input keys that a user has. So do I need to make 5 calls and then keep taking data from the previous output as the source for the next???
Other references I have looked at include: the manual
Thanks in advance,
Kindest Regards
Robin
This is a common misconception, with a compound array index key, it's still treated as a string, therefore the index key [2,10] is actually "[2,10]", and the index key [5,20], is actually "[5,20]".
So the reason that startkey=["2", "30"]shows the {"id":"5","key":["5","20"],"value":null}, row is because as a string it is > startkey.
Likewise, the Query startkey=[2,10]&endkey=[5,10] returns
{"total_rows":7,"rows":[
{"id":"2","key":[2,20],"value":null},
{"id":"3","key":[3,30],"value":null},
{"id":"4","key":[4,30],"value":null}
]
}
because startkey="[2,10]" < "[2,20]" && "[4,30]" < "[5,10]"=endkey, but "[5,20]" is not within that string Range.
Range Queries with startkey and endkey
startkey => endkey is a Range query using strcmp(), the group and group level is based on the string, where the comma is separating string tokens.
A Good Reference Link (since Couchbase Views work much like Apache CouchDB Views (inspired by them))
http://wiki.apache.org/couchdb/View_collation#Collation_Specification
Spatial View/Query
To achieve the result you are trying for, you could also write a Spatial View to have multi-dimensional Queries, numeric only. While you might not initially think of it
function (doc, meta) {
emit({
type: "Point",
coordinates: [doc.ID, doc.Price]
}, meta.id);
}
The Query would be a Bounding Box Query:
&bbox=2,0,8,30
{"total_rows":0,"rows":[
{"id":"2","bbox":[2,20,2,20],"geometry":{"type":"Point","coordinates":[2,20]},"value":"2"},
{"id":"3","bbox":[3,30,3,30],"geometry":{"type":"Point","coordinates":[3,30]},"value":"3"},
{"id":"4","bbox":[4,30,4,30],"geometry":{"type":"Point","coordinates":[4,30]},"value":"4"},
{"id":"5","bbox":[5,20,5,20],"geometry":{"type":"Point","coordinates":[5,20]},"value":"5"}
]
}
Another Query:
&bbox=2,30,8,30
{"total_rows":0,"rows":[
{"id":"3","bbox":[3,30,3,30],"geometry":{"type":"Point","coordinates":[3,30]},"value":"3"},
{"id":"4","bbox":[4,30,4,30],"geometry":{"type":"Point","coordinates":[4,30]},"value":"4"}
]
}