Building a JSON map for a self-referencing Ecto model - json

I have an Ecto model as such:
defmodule Project.Category do
use Project.Web, :model
schema "categories" do
field :name, :string
field :list_order, :integer
field :parent_id, :integer
belongs_to :menu, Project.Menu
has_many :subcategories, Project.Category, foreign_key: :parent_id
timestamps
end
#required_fields ~w(name list_order)
#optional_fields ~w(menu_id parent_id)
def changeset(model, params \\ :empty) do
model
|> cast(params, #required_fields, #optional_fields)
end
end
As you can see the Category model can reference itself via the subcategories atom.
Here is the view associated with this model:
defmodule Project.CategoryView do
use Project.Web, :view
def render("show.json", %{category: category}) do
json = %{
id: category.id,
name: category.name,
list_order: category.list_order
parent_id: category.parent_id
}
if is_list(category.subcategories) do
children = render_many(category.subcategories, Project.CategoryView, "show.json")
Map.put(json, :subcategories, children)
else
json
end
end
end
I have an if condition on subcategories so that I can play nice with Poison when they are not preloaded.
Finally, here are my 2 controller functions that invoke this view:
defmodule Project.CategoryController do
use Project.Web, :controller
alias Project.Category
def show(conn, %{"id" => id}) do
category = Repo.get!(Category, id)
render conn, "show.json", category: category
end
def showWithChildren(conn, %{"id" => id}) do
category = Repo.get!(Category, id)
|> Repo.preload [:subcategories, subcategories: :subcategories]
render conn, "show.json", category: category
end
end
The show function works fine:
{
"parent_id": null,
"name": "a",
"list_order": 4,
"id": 7
}
However, my showWithChildren function is limited to 2 levels of nesting because of how I use preloading:
{
"subcategories": [
{
"subcategories": [
{
"parent_id": 10,
"name": "d",
"list_order": 4,
"id": 11
}
],
"parent_id": 7,
"name": "c",
"list_order": 4,
"id": 10
},
{
"subcategories": [],
"parent_id": 7,
"name": "b",
"list_order": 9,
"id": 13
}
],
"parent_id": null,
"name": "a",
"list_order": 4,
"id": 7
}
For example, the category item 11 above also has subcategories but I am unable to reach them. Those subcategories can also have subcategories themselves, so the potential depth of the hierarchy is n.
I am aware that I need some recursive magic but since I'm new to both functional programming and Elixir, I cannot wrap my head around it. Any help is greatly appreciated.

You can consider doing the preloading in the view, so it works recursively:
def render("show.json", %{category: category}) do
%{id: category.id,
name: category.name,
list_order: category.list_order
parent_id: category.parent_id}
|> add_subcategories(category)
end
defp add_subcategories(json, %{subcategories: subcategories}) when is_list(subcategories) do
children =
subcategories
|> Repo.preload(:subcategories)
|> render_many(Project.CategoryView, "show.json")
Map.put(json, :subcategories, children)
end
defp add_subcategories(json, _category) do
json
end
Keep in mind this is not ideal for two reasons:
Ideally you don't want to do queries in views (but this is is recursive, so it is easier to piggyback in the view rendering)
You are going to emit multiple queries for the second level of subcategories
There is a book called SQL Antipatterns and, if I am not mistaken, it covers how to write tree structures. Your example is exposed as an antipattern in one of the free chapters. It is an excellent book and they explore solutions for all antipatterns.
PS: you want show_with_children and not showWithChildren.

Related

Querying to parent and children to a JSON format from MySQL 5.6?

I have a heirarchy of tables in a MySQL 5.6 database that I need to query to a JSON format for use by a javascript tree structure.
Just as a test in my flask I did the following for just the top level
def get_all_customers():
response_object = {'status': 'success'}
cnx = mysql.connector.connect(user="", password="", database="", host="localhost", port=3306)
cursor = cnx.cursor()
cursor.execute('SELECT idx, name FROM listcustomers ORDER BY name')
data = []
for idx, name in cursor:
data.append({'id': idx, 'label':name, 'otherProp': "Customer"})
response_object['customers'] = data
return jsonify(response_object)
which returns
[
{ id: 1,
label: "customer 1",
otherProp: "Customer"
},
...
]
But each customer has locations, and each location has areas, and each area has assets, and each asset has projects, and I need to also query them into children of this json object. So, for example, just going one level deeper to locations, I would need something like this -
[
{ id: 1,
label: "customer 1",
otherProp: "Customer",
children: [
{
id: 5,
label: "location 5",
otherProp: "Location"
},
...
]
},
...
]
where in my database listlocatiosn who links to listcustomers via the it's parentCustomerId column. How can I manage this? Eventually this tree will have about 13,000 objects so I know just querying the data and then parsing it with python would be far more inefficient than if I am able to query properly to begin with.

RoR, need to create recursive relationships in table associations

I'm very new to rails, and am a little stuck on the logic for this problem.
I have one table (using mysql) of employees, each of them with a manager_id key which refers to the employee they report to. So for example the employee with the title of "CEO" with an id of 1, has a manager_id of nil, and the employee with title of "CTO" has a manager_id of 1. So my records look like this
id: 1, first_name: "Bob", last_name: "Boss", title: "CEO", manager_id: null
id: 2, first_name: "Pat", last_name: "Guy", title: "CTO", manager_id: 1
id: 3, first_name: "John", last_name: "Dude", title: "VP of engineering", manager_id: 2
and my JSON structure should look like this
[
{id: 1, first_name: "Bob", last_name: "Boss", title: "CEO", manager_id: null, descendents: [
{id: 2, first_name: "Pat", last_name: "Guy", title: "CTO", manager_id: 1, descendents: [
{id: 3, first_name: "John", last_name: "Dude", title: "VP of engineering", manager_id: 2, descendents: [....]}
]},
{..more CEO descendents...}
]
I'm trying to create a nested JSON structure that starts at CEO, lists all employees that report to them, and each of those employees descendants. I was trying to write a script that creates this but I keep getting infinite recursive calls. This is what I have
#start at some root
#root = Employee.find_by title: 'CEO'
#convert to hash table
#results[0] = #root.attributes
#add direct_reports key
#results[0]["direct_reports"] = []
def getBelow(root=#root)
#reports = Employee.where("manager_id = ?", #root[:id])
if #reports.blank?
return []
else
#reports = #reports.map(&:attributes)
#reports.each do |person|
person["direct_reports"] = []
getBelow(person)
end
#reports = Employee.where("manager_id = ?", #root[:id])
root["direct_reports"] = #reports
end
return #root
end
#list = getBelow(#results[0])
If I'm passing in each new person object, shouldn't they all eventually end when #reports.blank? becomes true?
An alternative I was thinking of was to use table associations inspired by this blog post
https://hashrocket.com/blog/posts/recursive-sql-in-activerecord
but that seems a little too complicated.
Some issues in the getBelow method
You are always using #root, instead of using the param (root). So you are always starting again from the 'CEO'.
You are calling getBelow recursively but you are not using the result.
You call #reports = Employee.where("manager_id = ?", #root[:id]) twice.
You return #root.
As Jorge Najera said, there are gems that handle a tree structure easily. If you want to build it on your own, this is my suggestion:
#start at some root
#root = Employee.find_by manager_id: nil
#convert to hash table
#tree = #root.attributes
#add direct_reports key
#tree["direct_reports"] = getBelow(#root)
def getBelow(manager)
branch = []
Employee.where("manager_id = ?", manager.id).each do |employee|
node = employee.attributes
node["direct_reports"] = getBelow(employee)
branch << node
end
branch
end
This was not tested so I think you´ll get some errors, but I believe the idea is fine.

transform JSON to hashmap in Ruby

I am trying to convert a json file which contain object and array to a JSON file.
Below is the JSON file
{
"localbusiness":{
"name": "toto",
"phone": "+11234567890"
},
"date":"05/02/2016",
"time":"5:00pm",
"count":"4",
"userInfo":{
"name": "John Doe",
"phone": "+10987654321",
"email":"john.doe#unknown.com",
"userId":"user1234333"
}
}
my goal is to save this is a database such as MongoId. I would like to use map to get something like:
localbusiness_name => "toto",
localbusiness_phone => "+11234567890",
date => "05/02/2016",
...
userInfo_name => "John Doe"
...
I have tried map but it's not splitting the array of local business or userInfo
def format_entry
ps = #params.map do | h |
ps.merge!(h)
##logger.info("entry #{h}")
end
##logger.info("formatting the data #{ps}")
ps
end
I do not really how to parse each entry and rebuild the name
It looks like to me you are trying to "flatten" the inner hashes into one big hash. Flatten being incorrect because you want to prepend the hash's key to the sub-hash's key. This will require looping through the hash, and then looping again through each sub hash. This code example will only work if you have 1 layer deep. if you have multiple layers, then I would suggest making two methods, or a recursive method.
#business = { # This is a hash, not a json blob, but you can take json and call JSON.parse(blob) to turn it into a hash.
"localbusiness":{
"name": "toto",
"phone": "+11234567890"
},
"date":"05/02/2016",
"time":"5:00pm",
"count":"4",
"userInfo":{
"name": "John Doe",
"phone": "+10987654321",
"email":"john.doe#unknown.com",
"userId":"user1234333"
}
}
#squashed_business = Hash.new
#business.each do |k, v|
if v.is_a? Hash
v.each do |key, value|
#squashed_business.merge! (k.to_s + "_" + key.to_s) => value
end
else
#squashed_business.merge! k => v
end
end
I noticed that you are getting "unexpected" outcomes when enumerating over a hash #params.each { |h| ... } because it gives you both a key and a value. Instead you want to do #params.each { |key, value| ... } as I did in the above code example.

Limit fields of JSON response

I'm using the PhoenixFramework and the library Poison.
Current I'm working on an REST API. Now I need to encode the model Book in two different ways.
In a list of all books with only base informations (GET /books)
In a detailed view with all informations (GET /book/1)
GET /books
{
"books": [
{
"id": 1,
"name": "Book one"
},
{
"id": 2,
"name": "Book two"
},
// ...
]
}
GET /books/1
{
"book": {
"id": 1,
"name": "Book one",
"description": "This is book one.",
"author": "Max Mustermann",
// ...
}
}
Encoder of Book
defimpl Poison.Encoder, for: MyProject.Book do
#attributes [:id, :name, :description, :author]
def encode(project, _options) do
project
|> Map.take(#attributes)
|> Poison.encode!
end
end
Snippet controller
def index(conn, _params) do
books = Repo.all(Book)
json(conn, %{books: books})
end
How to limit the fields? I search for a option like :only or :exclude.
Has anyone experience with this problem?
Thanks for help!
You can use render_many/4 and render_one/4:
defmodule MyApp.BookView do
def render("index.json", %{books: books}) do
render_many(books, __MODULE__, "book.json")
end
def render("show.json", %{book: book}) do
render_one(book, __MODULE__, "full_book.json")
end
def render("book.json", %{book: book}) do
%{
id: book.id,
name: book.name
}
end
def render("full_book.json", %{book: book}) do
%{
id: book.id,
name: book.name,
description: description,
author: book.author
...
}
end
end
Please note that the name in assigns (2nd argument of render) is determined by the module. See Render many to many relationship JSON in Phoenix Framework for an example of using a different name.
You can then call this from your controller using:
render(conn, "index.json", books: Repo.all(Book))
render(conn, "show.json", book: Repo.get(Book, 1))

Postgres JSON data type Rails query

I am using Postgres' json data type but want to do a query/ordering with data that is nested within the json.
I want to order or query with .where on the json data type. For example, I want to query for users that have a follower count > 500 or I want to order by follower or following count.
Thanks!
Example:
model User
data: {
"photos"=>[
{"type"=>"facebook", "type_id"=>"facebook", "type_name"=>"Facebook", "url"=>"facebook.com"}
],
"social_profiles"=>[
{"type"=>"vimeo", "type_id"=>"vimeo", "type_name"=>"Vimeo", "url"=>"http://vimeo.com/", "username"=>"v", "id"=>"1"},
{"bio"=>"I am not a person, but a series of plants", "followers"=>1500, "following"=>240, "type"=>"twitter", "type_id"=>"twitter", "type_name"=>"Twitter", "url"=>"http://www.twitter.com/", "username"=>"123", "id"=>"123"}
]
}
For any who stumbles upon this. I have come up with a list of queries using ActiveRecord and Postgres' JSON data type. Feel free to edit this to make it more clear.
Documentation to the JSON operators used below: https://www.postgresql.org/docs/current/functions-json.html.
# Sort based on the Hstore data:
Post.order("data->'hello' DESC")
=> #<ActiveRecord::Relation [
#<Post id: 4, data: {"hi"=>"23", "hello"=>"22"}>,
#<Post id: 3, data: {"hi"=>"13", "hello"=>"21"}>,
#<Post id: 2, data: {"hi"=>"3", "hello"=>"2"}>,
#<Post id: 1, data: {"hi"=>"2", "hello"=>"1"}>]>
# Where inside a JSON object:
Record.where("data ->> 'likelihood' = '0.89'")
# Example json object:
r.column_data
=> {"data1"=>[1, 2, 3],
"data2"=>"data2-3",
"array"=>[{"hello"=>1}, {"hi"=>2}],
"nest"=>{"nest1"=>"yes"}}
# Nested search:
Record.where("column_data -> 'nest' ->> 'nest1' = 'yes' ")
# Search within array:
Record.where("column_data #>> '{data1,1}' = '2' ")
# Search within a value that's an array:
Record.where("column_data #> '{array,0}' ->> 'hello' = '1' ")
# this only find for one element of the array.
# All elements:
Record.where("column_data ->> 'array' LIKE '%hello%' ") # bad
Record.where("column_data ->> 'array' LIKE ?", "%hello%") # good
According to this http://edgeguides.rubyonrails.org/active_record_postgresql.html#json
there's a difference in using -> and ->>:
# db/migrate/20131220144913_create_events.rb
create_table :events do |t|
t.json 'payload'
end
# app/models/event.rb
class Event < ActiveRecord::Base
end
# Usage
Event.create(payload: { kind: "user_renamed", change: ["jack", "john"]})
event = Event.first
event.payload # => {"kind"=>"user_renamed", "change"=>["jack", "john"]}
## Query based on JSON document
# The -> operator returns the original JSON type (which might be an object), whereas ->> returns text
Event.where("payload->>'kind' = ?", "user_renamed")
So you should try Record.where("data ->> 'status' = 200 ") or the operator that suits your query (http://www.postgresql.org/docs/current/static/functions-json.html).
Your question doesn't seem to correspond to the data you've shown, but if your table is named users and data is a field in that table with JSON like {count:123}, then the query
SELECT * WHERE data->'count' > 500 FROM users
will work. Take a look at your database schema to make sure you understand the layout and check that the query works before complicating it with Rails conventions.
JSON filtering in Rails
Event.create( payload: [{ "name": 'Jack', "age": 12 },
{ "name": 'John', "age": 13 },
{ "name": 'Dohn', "age": 24 }]
Event.where('payload #> ?', '[{"age": 12}]')
#You can also filter by name key
Event.where('payload #> ?', '[{"name": "John"}]')
#You can also filter by {"name":"Jack", "age":12}
Event.where('payload #> ?', {"name":"Jack", "age":12}.to_json)
You can find more about this here