Sending information to a widget in Dashing - mysql

I'm creating a job in Ruby which takes info from a mysql database and sends it to a html page to create a widget. The thing is, I can't get the data to be displayed when I take it out of a mysql database, however when I type the input in manually and send that to the html page...it works!
require 'Mysql2'
SCHEDULER.every '10s', :first_in => 0 do |job|
$application, $details, $timestamp = 0
table = Array.new
client = Mysql2::Client.new(:host => "localhost", :username => "****", :password => "****", :database => "Sample_Database")
#Gets statistics for latest input
sstc = client.query("SELECT * FROM Probably_Serious_Alerts")
sstc.each do |row|
$row = "{ cols: [ {value: '" + row["time_stamp"].to_s + "'}, {value: '"+ row["application"].to_s + "'}, {value: '" + row["details"].to_s + "'}]}"
table.push($row)
end
rows = "rows = ["
for row in table
rows = rows + row + ","
end
rows = rows[0...-1]
rows = rows + ']'
hrows = [
{ cols: [ {value: 'Time'}, {value: 'Monitoring System'}, {value: 'Event'}]}
]
sleep(1)
#rows = [
# { cols: [ {value: '2016-09-19 14:39:30 +0100'}, {value: 'Solarwinds'}, {value: 'First Solarwinds Error'}]},
# { cols: [ {value: '2016-09-19 15:24:17 +0100'}, {value: 'Nagios'}, {value: 'First Nagios'}]}]
puts rows
send_event('my-table', { hrows: hrows, rows: rows } )
end
The commented code is the code which is printed to the terminal when I run it. This is also what I send to the HTML page and works on the widget. Any ideas? Is something async?

I know it's a bit late but since I stumbled upon this question on Stackoverflow when Googling for the same issue I had when trying to push data to the Smashing Table Widget I thought someone else might be happy if I shared my solution. The issue in your code is that you put the rows object you return as a string, however the Table Widget expects, let me quote from the Documentation:
To send a row to the tbody (or thead), send a array of hashes to rows (hrows for thead). The bindings work from row to column. Every column should be it's own array element off a cols hash. The hash must have a key of "value" in order to show up. To send multiple rows, use an array of these hashes.
So you have to transform the MySQL Result into an Array of Hashes. See my code below, note that "Loc Nr.", "Address" and "Status" are the Column Names of my MySQL Table:
results = db.query(sql)
hrows = [
{ cols: [ {value: 'Loc Nr.'}, {value: 'Address'}, {value: 'Status'}]}
]
table = Array.new
results.each do |row|
table.push({ cols: [ {value: row['Loc Nr.']}, {value: row['Address']}, {value: row['Status']}]})
end
send_event('locations', { hrows: hrows, rows: table } )
Hope you already found the solution but it might help someone else :D

Related

Creating json with RUBY looping through SQL Server table

This is a followup to this question:
Ruby create JSON from SQL Server
I was able to create nested arrays in JSON. But I'm struggling with looping through records and appending a file with each record. Also how would I add a root element just at the top of the json and not on each record. "aaSequences" needs to be at the top just once... I also need a comma between each record.
here is my code so far
require 'pp'
require 'tiny_tds'
require 'awesome_print'
require 'json'
class Document
def initialize strategy
#document = strategy
#load helper functions
load "helpers_ruby.rb"
#set environment 'dev', 'qa', or 'production'
load "envconfig_ruby.rb"
end
def StartUP
#document.StartUP
end
def getseqrecord
#document.getseqrecord
end
end
class GetSqlaaSequence
def StartUP
##system "clear" ##linux
system "cls" ##Windows
# create connection to db
$connReportingDB = createReportingxxSqlConn($ms_sql_host, $ms_sql_user, $ms_sql_password, $ms_sql_dbname)
##$currentDateTime = DateTime.now
##pp 'def StartUP ran at: '+$currentDateTime.to_s
end
def getseqrecord
# get the aaaaSequences data
#result = $connReportingDB.execute("SELECT
[jsonFile]
,[id]
,[title]
,[authorIds]
,[name]
,[aminoAcids]
,[schemaId]
,[registryId]
,[namingStrategy]
FROM tablename
")
$aaSequences = Array.new
#i = 0
#result.each do |aaSequence|
jsonFile = aaSequence['jsonFile']
id = aaSequence['id']
title = aaSequence['title']
authorIds = aaSequence['authorIds']
name = aaSequence['name']
aminoAcids = aaSequence['aminoAcids']
schemaId = aaSequence['schemaId']
registryId = aaSequence['registryId']
namingStrategy = aaSequence['namingStrategy']
##end
#hash = Hash[
"jsonFile", jsonFile,
"id", id,
"title", title,
"authorIds", authorIds,
"name", name,
"aminoAcids", aminoAcids,
"schemaId", schemaId,
"registryId", registryId,
"namingStrategy", namingStrategy
]
#filename = jsonFile
jsonFileOutput0 = {:"#{title}" => [{:authorIds => ["#{authorIds}"],:aminoAcids => "#{aminoAcids}",:name => "#{name}",:schemaId => "#{schemaId}",:registryId => "#{registryId}",:namingStrategy => "#{namingStrategy}"}]}
jsonFileOutput = JSON.pretty_generate(jsonFileOutput0)
File.open(jsonFile,"a") do |f|
f.write(jsonFileOutput)
####ad the comma between records...Not sure if this is the best way to do it...
# File.open(jsonFile,"a") do |f|
# f.write(',')
# end
end
$aaSequences[#i] = #hash
#i = #i + 1
###createReportingSqlConn.close
end
end
end
Document.new(GetSqlaaSequence.new).StartUP
#get aaSequences and create json files
Document.new(GetSqlaaSequence.new).getseqrecord
here is a sample of the json it creates so far...
{
"aaSequences": [
{
"authorIds": [
"fff_fdfdfdfd"
],
"aminoAcids": "aminoAcids_data",
"name": "fdfdfddf-555_1",
"schemaId": "5555fdfd5",
"registryId": "5fdfdfdf",
"namingStrategy": "NEW_IDS"
}
]
}{
"aaSequences": [
{
"authorIds": [
"fff_fdfdfdfd"
],
"aminoAcids": "aminoAcids_data",
"name": "fdfdfddf-555_2",
"schemaId": "5555fdfd5",
"registryId": "5fdfdfdf",
"namingStrategy": "NEW_IDS"
}
]
}
and here is an example of what I need it to look like
{
"aaSequences": [
{
"authorIds": [
"authorIds_data"
],
"aminoAcids": "aminoAcids_data",
"name": "name_data",
"schemaId": "schemaId_data",
"registryId": "registryId_data",
"namingStrategy": "namingStrategy_data"
},
{
"authorIds": [
"authorIds_data"
],
"aminoAcids": "aminoAcids_data",
"name": "name_data",
"schemaId": "schemaId_data",
"registryId": "registryId_data",
"namingStrategy": "namingStrategy_data"
}
]
}
You can just do the whole thing in SQL using FOR JSON.
Unfortunately, arrays are not possible using this method. There are anumber of hacks, but the easiest one in your situation is to just append to [] using JSON_MODIFY
SELECT
authorIds = JSON_MODIFY('[]', 'append $', a.authorIds),
[aminoAcids],
[name],
[schemaId],
[registryId],
[namingStrategy]
FROM aaSequences a
FOR JSON PATH, ROOT('aaSequences');
db<>fiddle

Querying to parent and children to a JSON format from MySQL 5.6?

I have a heirarchy of tables in a MySQL 5.6 database that I need to query to a JSON format for use by a javascript tree structure.
Just as a test in my flask I did the following for just the top level
def get_all_customers():
response_object = {'status': 'success'}
cnx = mysql.connector.connect(user="", password="", database="", host="localhost", port=3306)
cursor = cnx.cursor()
cursor.execute('SELECT idx, name FROM listcustomers ORDER BY name')
data = []
for idx, name in cursor:
data.append({'id': idx, 'label':name, 'otherProp': "Customer"})
response_object['customers'] = data
return jsonify(response_object)
which returns
[
{ id: 1,
label: "customer 1",
otherProp: "Customer"
},
...
]
But each customer has locations, and each location has areas, and each area has assets, and each asset has projects, and I need to also query them into children of this json object. So, for example, just going one level deeper to locations, I would need something like this -
[
{ id: 1,
label: "customer 1",
otherProp: "Customer",
children: [
{
id: 5,
label: "location 5",
otherProp: "Location"
},
...
]
},
...
]
where in my database listlocatiosn who links to listcustomers via the it's parentCustomerId column. How can I manage this? Eventually this tree will have about 13,000 objects so I know just querying the data and then parsing it with python would be far more inefficient than if I am able to query properly to begin with.

Retrieving array from MySQL

I'm not using sqlite3 gem
I'm using mysql2 gem
I'm retrieving data from MySQL database given that it meets the condition of a certain event type and severity. However, it returns only one row instead of an array of results. It really puzzles me. Shouldnt .map return an array?
result = connect.query("SELECT * FROM data WHERE event_type = 'ALARM_OPENED' AND severity = '2'")
equipments = result.map do |record|
[
record['sourcetime'].strftime('%H:%M:%S'),
record['equipment_id'],
record['description']
]
end
p equipments
I had misread your question...I think what you are looking for is in here.
UPDATE
You can use each instead, like this:
#!/usr/bin/env ruby
require 'mysql2'
connect= Mysql2::Client.new(:host => '', :username => '', :password => '', :database => '')
equipments = []
result = connect.query("SELECT * FROM data WHERE event_type = 'ALARM_OPENED' AND severity = '2'", :symbolize_keys => true).each do |row|
equipments << [
row[:sourcetime].strftime('%H:%M:%S'),
row[:equipment_id],
row[:description]
]
end
puts "#equipments {equipments}"
EDITED:
I forgot to add .each at the end of the query. So it was returning the initialized empty array instead.
You must need to change your sql statement :
result = connect.query("SELECT * FROM data WHERE event_type = 'ALARM_OPENED' AND severity = '2'", :as => :array)

Postgres JSON data type Rails query

I am using Postgres' json data type but want to do a query/ordering with data that is nested within the json.
I want to order or query with .where on the json data type. For example, I want to query for users that have a follower count > 500 or I want to order by follower or following count.
Thanks!
Example:
model User
data: {
"photos"=>[
{"type"=>"facebook", "type_id"=>"facebook", "type_name"=>"Facebook", "url"=>"facebook.com"}
],
"social_profiles"=>[
{"type"=>"vimeo", "type_id"=>"vimeo", "type_name"=>"Vimeo", "url"=>"http://vimeo.com/", "username"=>"v", "id"=>"1"},
{"bio"=>"I am not a person, but a series of plants", "followers"=>1500, "following"=>240, "type"=>"twitter", "type_id"=>"twitter", "type_name"=>"Twitter", "url"=>"http://www.twitter.com/", "username"=>"123", "id"=>"123"}
]
}
For any who stumbles upon this. I have come up with a list of queries using ActiveRecord and Postgres' JSON data type. Feel free to edit this to make it more clear.
Documentation to the JSON operators used below: https://www.postgresql.org/docs/current/functions-json.html.
# Sort based on the Hstore data:
Post.order("data->'hello' DESC")
=> #<ActiveRecord::Relation [
#<Post id: 4, data: {"hi"=>"23", "hello"=>"22"}>,
#<Post id: 3, data: {"hi"=>"13", "hello"=>"21"}>,
#<Post id: 2, data: {"hi"=>"3", "hello"=>"2"}>,
#<Post id: 1, data: {"hi"=>"2", "hello"=>"1"}>]>
# Where inside a JSON object:
Record.where("data ->> 'likelihood' = '0.89'")
# Example json object:
r.column_data
=> {"data1"=>[1, 2, 3],
"data2"=>"data2-3",
"array"=>[{"hello"=>1}, {"hi"=>2}],
"nest"=>{"nest1"=>"yes"}}
# Nested search:
Record.where("column_data -> 'nest' ->> 'nest1' = 'yes' ")
# Search within array:
Record.where("column_data #>> '{data1,1}' = '2' ")
# Search within a value that's an array:
Record.where("column_data #> '{array,0}' ->> 'hello' = '1' ")
# this only find for one element of the array.
# All elements:
Record.where("column_data ->> 'array' LIKE '%hello%' ") # bad
Record.where("column_data ->> 'array' LIKE ?", "%hello%") # good
According to this http://edgeguides.rubyonrails.org/active_record_postgresql.html#json
there's a difference in using -> and ->>:
# db/migrate/20131220144913_create_events.rb
create_table :events do |t|
t.json 'payload'
end
# app/models/event.rb
class Event < ActiveRecord::Base
end
# Usage
Event.create(payload: { kind: "user_renamed", change: ["jack", "john"]})
event = Event.first
event.payload # => {"kind"=>"user_renamed", "change"=>["jack", "john"]}
## Query based on JSON document
# The -> operator returns the original JSON type (which might be an object), whereas ->> returns text
Event.where("payload->>'kind' = ?", "user_renamed")
So you should try Record.where("data ->> 'status' = 200 ") or the operator that suits your query (http://www.postgresql.org/docs/current/static/functions-json.html).
Your question doesn't seem to correspond to the data you've shown, but if your table is named users and data is a field in that table with JSON like {count:123}, then the query
SELECT * WHERE data->'count' > 500 FROM users
will work. Take a look at your database schema to make sure you understand the layout and check that the query works before complicating it with Rails conventions.
JSON filtering in Rails
Event.create( payload: [{ "name": 'Jack', "age": 12 },
{ "name": 'John', "age": 13 },
{ "name": 'Dohn', "age": 24 }]
Event.where('payload #> ?', '[{"age": 12}]')
#You can also filter by name key
Event.where('payload #> ?', '[{"name": "John"}]')
#You can also filter by {"name":"Jack", "age":12}
Event.where('payload #> ?', {"name":"Jack", "age":12}.to_json)
You can find more about this here

How to extract nested hashes into database table?

I am trying to extract some data that is being returned in a nested hash from Mailchimp. Here is a trimmed down version of the results I am getting. For each email there are multiple GROUPINGS and for each GROUPING there are multiple GROUPS.
My objective is to get this into a mysql table with a layout like: email_addr, list, grouping1_id, grouping1_name, group1_name, group1_interest, group2_name, group2_interest, grouping2_id, grouping2_name, etc. So there is one row per subscriber with all the grouping and group information.
{"email"=>"dummy#gmail.com", "merges"=>{"EMAIL"=>"dummy#gmail.com",
"GROUPINGS"=>[{"id"=>1, "name"=>"Grouping One", "groups"=>[{"name"=>"Group One",
"interested"=>false}, {"name"=>"Group", "interested"=>true},
{"name"=>"Group Three", "interested"=>true}]}, {"id"=>2, "name"=>"Grouping Two",
"groups"=>[{"name"=>"Group Four", "interested"=>false},
{"name"=>"Group Five", "interested"=>false}]}]}}
Right now, the code I have below runs and inserts the results of the nested blocks into the table but there is one row for each pass through the groups.each_with_index statement. My approach so far seems overly complicated but I am not sure of how to approach this to process the data correctly.
Any help is appreciated.
UPDATED:
I cleaned up the logic a bit and separated the database writes into each level of the hash processing. Now the data is inserted and updated in the database correctly. Although this still feels very inelegant.
def organize_members_subs
#members_data = #members_subs["data"]
#members_data.each do |member|
#email_addr = member["email"]
#db.query("INSERT INTO db.details
(email_addr, list)
VALUES ('#{#email_addr}', '#{#list}' ) ")
groupings = member["merges"]["GROUPINGS"]
groupings.each_with_index do |grouping, index|
#groupings_name = grouping["name"]
#groupings_id = grouping["id"]
#groupings_label = "grp#{index}_"
#db.query("UPDATE db.details
SET grouping#{index}_id = '#{#groupings_id}'
, grouping#{index}_name = '#{#groupings_name}'
WHERE email_addr = '#{#email_addr}' ")
groups = member["merges"]["GROUPINGS"][index]["groups"]
groups.each_with_index do |group, index|
#group_name = group["name"]
#group_interested = group["interested"]
#db.query("UPDATE db.details
SET #{#groupings_label}group#{index}_name = '#{#group_name}'
, #{#groupings_label}group#{index}_int = '#{#group_interested}'
WHERE email_addr = '#{#email_addr}' ")
break if index == groups.length
end
break if index == groupings.length
end
end
end
To start, I wanted to take a closer at look your hash. Rather than reformatting it myself, I did this:
require "awesome_print"
h = `{"email"=>..., "interested"=>false}]}]}}`
ap h
Scroll down to the bottom of my answer to see ap's formatting of the hash.
I will answer your question assuming the db structure is a given, but would like to make a few points:
If "id" is unique for each grouping record, could you make that the key, and dispense with index?
If "name" is unique for each grouping record, could you dispense with both "id" and index?
If "name" is unique for each group record (for a given grouping), could you just have group["name"] => group["interested"] for each group?
Moving on to your code, I will also assume the structure of your hash is given. Later, I will revisit that assumption.
The changes I propose to your code are fairly minor and some are purely stylistic:
Make all instance variables local variables, meaning that two additional arguments must be passed to def organize_members_subs.
With two possible exceptions, eliminate local variables that are only used once after being defined. For example, rather than
groupings_id = grouping["id"], then SET grouping#{index}_id = '#{#groupings_id}', just have SET grouping#{index}_id = '#{grouping["id"]}'.
The two possible exceptions are groupings and groups. For example, you could get rid of the former by writing
member["merges"]["GROUPINGS"].each_with_index do |grouping, index_1|. I'd keep them as variables (so I could easily check their values),
but that's a stylistic decision.
The variable index in groupings.each_with_index do |grouping, index| is in scope within the inner block, which uses an iterator variable with the same name.
I presume that the latter takes precedence, but they should be named differently. I've changed them to index_out and index_in, respectively.
index_out ranges from 0 to groupings.length-1, so break if index_out == groupings.length will never be executed, and therefore may be removed. Ditto for break if index_in == groups.length.
I moved groupings_label = "grp#{index}_" down to draw attention to the fact that it is needed only later, not in the preceding SET expression.
These changes result in the following:
def organize_members_subs(db, list, #members_subs["data"])
members_data.each do |member|
email_addr = member["email"]
db.query("INSERT INTO db.details
(email_addr, list)
VALUES ('#{email_addr}', '#{list}' ) ")
groupings = member["merges"]["GROUPINGS"]
groupings.each_with_index do |grouping, index_out|
db.query("UPDATE db.details
SET grouping#{index_out}_id = '#{grouping["id"]}'
, grouping#{index_out}_name = '#{grouping["name"]}'
WHERE email_addr = '#{email_addr}' ")
groupings_label = "grp#{index_out}_"
groups = member["merges"]["GROUPINGS"][index_out]["groups"]
groups.each_with_index do |group, index_in|
db.query("UPDATE db.details
SET #{groupings_label}group#{index_in}_name = '#{group["name"]}'
, #{groupings_label}group#{index_in}_int = '#{group["interested"]}'
WHERE email_addr = '#{email_addr}' ")
end
end
end
end
Looking at your hash, I am wondering if you could simplify it to the following (formatting courtesy of awesome print):
{
"email" => "dummy#gmail.com",
"merges" => {
"EMAIL" => "dummy#gmail.com",
"GROUPINGS" => {
1 => {
"name" => "Grouping One",
"groups" => {
"Group One" => false,
"Group Two" => true,
"Group Three" => true
}
},
2 => {
"name" => "Grouping Two",
"groups" => {
"Group Four" => false,
"Group Five" => false
}
}
}
}
}
or even
{
"email" => "dummy#gmail.com",
"merges" => {
"EMAIL" => "dummy#gmail.com",
"GROUPINGS" => {
"Grouping One" => {
"Group One" => false,
"Group Two" => true,
"Group Three" => true
},
"Grouping Two" => {
"Group Four" => false,
"Group Five" => false
}
}
}
}
These are not so much as suggestions, but just food for thought.
Awesome print applied to your hash:
ap h # =>
{
"email" => "dummy#gmail.com",
"merges" => {
"EMAIL" => "dummy#gmail.com",
"GROUPINGS" => [
[0] {
"id" => 1,
"name" => "Grouping One",
"groups" => [
[0] {
"name" => "Group One",
"interested" => false
},
[1] {
"name" => "Group",
"interested" => true
},
[2] {
"name" => "Group Three",
"interested" => true
}
]
},
[1] {
"id" => 2,
"name" => "Grouping Two",
"groups" => [
[0] {
"name" => "Group Four",
"interested" => false
},
[1] {
"name" => "Group Five",
"interested" => false
}
]
}
]
}
}
First, maybe extra, but I like to work with symbols since I do a lot of my work in Rails. So let's steal a method from here: How do I convert a Ruby hash so that all of its keys are symbols?
def recursive_symbolize_keys(h)
case h
when Hash
Hash[
h.map do |k, v|
[ k.respond_to?(:to_sym) ? k.to_sym : k, recursive_symbolize_keys(v) ]
end
]
when Enumerable
h.map { |v| recursive_symbolize_keys(v) }
else
h
end
end
OK, lets build a class to make this easier to manipulate and extend as our needs change:
class MemberSub
attr_accessor :email, :groupings, :data_hash, :list, :data_row, :db_sql
def initialize(data_hash)
#convert all keys to symbols
#data_hash = recursive_symbolize_keys(data_hash)
#email = #data_hash[:email]
#list = 'Members'
#groupings = #data_hash[:merges][:GROUPINGS]
#data_row = data_row
#db_sql = db_insert
end
def data_row
#returns a data row for DB
row_hash = {}
row_hash['email'] = #email
row_hash['list'] = #list
gc = 1
#iterate through groupings
#groupings.each_with_index do |grouping, index|
row_hash["grouping#{index + 1}_id"] = grouping[:id]
row_hash["grouping#{index + 1}_name"] = grouping[:name]
#iterate through the groups
grouping[:groups].each do |group|
row_hash["group#{gc}_name"] = group[:name]
row_hash["group#{gc}_interest"] = group[:interested]
gc += 1
end
end
row_hash
end
def db_insert
"INSERT INTO db.details (#{#data_row.keys}) VALUES (#{#data_row.values})".tr('[]','')
end
end
Now you can do feed it a row using what ever iteration method and make a new object:
row = MemberSub.new({"email"=>"dummy#gmail.com", "list"=>"Members", "merges"=>
{"EMAIL"=>"dummy#gmail.com", "GROUPINGS"=>[{"id"=>1, "name"=>"Grouping One", "groups"=>
[{"name"=>"Group One", "interested"=>false}, {"name"=>"Group Two", "interested"=>true},
{"name"=>"Group Three", "interested"=>true}]}, {"id"=>2, "name"=>"Grouping Two", "groups"=>
[{"name"=>"Group Four", "interested"=>false}, {"name"=>"Group Five", "interested"=>false}]}]}})
and make a query:
db.query(row.db_sql)
db.query(INSERT INTO db.details ("email", "list", "grouping1_id", "grouping1_name",
"group1_name", "group1_interest", "group2_name", "group2_interest", "group3_name",
"group3_interest", "grouping2_id", "grouping2_name", "group4_name", "group4_interest",
"group5_name", "group5_interest") VALUES ("dummy#gmail.com", "Members", 1, "Grouping One",
"Group One", false, "Group Two", true, "Group Three", true, 2, "Grouping Two", "Group Four",
false, "Group Five", false))
The other methods should be self explanatory. You don't have to have them all available as attar_accessor but I just did that for example.