Just to level-set:
CakePHP 3 introduces Entity objects which can represent an ORM (database) record as an object rather than an array. Creating the object from the raw data is called "hydration". This has pros and cons, depending on what you're trying to achieve, so CakePHP gives you the option to control hydration through the hydrate() function which can be chained in the query.
What I've observed is only the top level results are hydrated; nested results are not. So if my query is something like:
$authors=$this->Authors->find("all")->contain("Books");
$this->set("authors",$authors);
will return something along the lines of
authors (array) << This is an array since we can have multiple records
0 (object) <<< This is the Entity object representing the first Author
id 1
name "Roger Kaplan"
[other author fields]
books (array) <<< This is an array because there are multiple books
0 (array) <<< I expect this to be an entity object!!
id 100
title "CakePHP Made Easy"
[other book fields]
1 (array) <<< I want this to be an entity too
id 101
title "Solving Java-induced Neuroses"
[other book fields]
Is it possible to have the nested entities hydrated as well?
The reason I'm asking is that I'm building helpers which expect an entity object to be passed, and uses the metadata on the entity object to do interesting things. I want to be able to pass nested records as well as top-level ones.
EDIT:
Something I just noticed is that belongsTo associations, which will only contain one record, will be inserted as an array of values (ie not an entity) while hasMany associations will return an array of entities. Here is the dump from my actual project; I've attempted to edit it down for clarity:
$message = $this->Messages->find("all")->where(["Messages.id" => $message_id])->contain(["MessageBodies","JobOrders","Candidates"])->first();
Messages belongsTo Candidates and JobOrders, and hasMany MessageBodies.
Here is a rendering of the result:
message(array)
id 1
job_order_id 2
candidate_id 1
candidate(array)
id 1
first_name Roger
last_name Kaplan
job_order(array)
id 2
name Chief Cook and Bottle Washer
message_bodies(array)
0(object)
1(object)
2(object)
So if my assumption is correct, that only hasMany associations are returned as an array of entities, the question is, how can I get belongsTo (and possibly hasOne, which I tend not to use) contained data to show up as Entity's?
The data passed into the view does indeed contain nested entities, as verified by the debugger. I was using the variable viewer in the Cake toolbar to look into the structure, and that view was reporting the embedded entities as arrays. Based on that data, I was reaching into the structure with ["array syntax"] but the Cake entity is smart enough to intercept that and convert to get() calls.
Related
Does Ruby/Rails have any magic logic that can create an object (Customer.create(...)) from a JSON or YAML with the defined parameters, along with extraneous key-values in said file?
The context of the problem is ingesting/seeding a table Customer in a Rails app from a bulk JSON file. The model contains only pertinent information but will evolve to include other information (like "birth-date", "gender").
(And for further but unnecessary context -- one view will display information about all 'Customer' when passed a list of customer names or IDs.)
For instance given a model,
class Customer:
def initialize(first, last, age)
#cust_first = first
#cust_last = last
#cust_age = age
end
...
end
...is there a mechanic where I can just feed a huge JSON, and it will feed it in? That way as the model Customer transforms, the ingestion script does not need to be changed?
# [{'First':'John','Last':'Smith','Age':'50','Gender':'M','ActiveSub':'N'...},{...},...]
bulk_data = JSON.parse(File.read('tmp/customers-20221001.json'))
bulk_data.each do |customer_json|
temp = Customer.new(customer_json) # <-- syntax in question
temp.save
end
Also interested in why this idea might be unsafe, messy, etc.
I apologize ahead of time because I don't even know how to ask this question. I am using FormStack and VB to get a list of Health Care Providers; up to 20. The way FormStack is returning this is as one object where each field is uniquely identified. Right now I have a class setup like so:
<JsonProperty("51791254")>
Property facility_name_01 As String
<JsonProperty("51791260")>
Property doctor_first_name_01 As String
<JsonProperty("51791294")>
Property doctor_last_name_01 As String
<JsonProperty("51791347")>
Property facility_address_1_01 As String
...
<JsonProperty("51792559")>
Property add_another_health_care_provider_01 As String
<JsonProperty("51792598")>
Property facility_name_02 As String
<JsonProperty("51792599")>
Property doctor_first_name_02 As String
<JsonProperty("51792600")>
Property doctor_last_name_02 As String
<JsonProperty("51792601")>
Property facility_address_1_02 As String
...
<JsonProperty("51792613")>
Property add_another_health_care_provider_02 As String
This list repeats to ..._20 for each health care provider. What I would like is to be able to loop through the json and create a list of up to 20 Health Care Providers if data exists. I guess my question; is there a more direct/better approach instead of creating a class with 344 properties that I then loop through and create a list of 20 HealthCareProividers with 17 properties, (the 4 extra fields are id fields). Again, I apologize about my ignorance in this area and appreciate any help and clarification.
I have been thrown in to the deep end and there is no life guard. More accurately, I am the life guard and haven't learned how to swim yet.
Sorry I did not include the JSON and thank you for the responses thus far
{"51791254":"HIllcrest Hoispital","51791260":"","51791294":"","51791347":"1234 Mayfield Rd","51791352":"","51791356":"Mayfield","51791361":"Ohio","51791384":"44124","51791388":"","51791397":"","51791399":"12/19/1974","51791415":"04/17/2017","51791510":"Treating","51791582":"106 | General Medicine","51791589":"Test entry for test client","51792559":"Yes","51792598":"Lake West","51792599":"","51792600":"","51792601":"1234 Euclid Ave","51792602":"","51792603":"Willoughby","51792604":"Ohio","51792605":"44092","51792606":"","51792607":"","51792608":"10/13/2003","51792609":"04/17/2017","51792610":"Diagnosing","51792611":"106 | General Medicine","51792612":"Second Test ","51792613":"Yes","51792970":"Rite Aid","51792971":"","51792972":"","51792973":"1234 SOM Center Rd","51792974":"","51792975":"Willoughby","51792976":"Ohio","51792977":"44092","51792978":"","51792979":"","51792980":"12/17/2013","51792981":"04/17/2017","51792982":"Pharmacy","51792983":"1 | Not Specified","51792984":"Test Pharmacy","51793056":"","51793057":"","51792876":"","51792877":"","51793033":"","51793034":"","51793015":"","51793016":"","51792997":"","51792998":"","51792963":"","51792964":"","51792945":"","51792946":"","51792928":"","51792929":"","51792911":"","51792912":"","51792893":"","51792894":"","51792753":"","51792754":"","51792859":"","51792860":"","51792842":"","51792843":"","51792825":"","51792826":"","51792808":"","51792809":"","51792791":"","51792792":"","51792774":"","51792775":"","51793552":"CFC3238D6271E613","51793634":"04/17/2017","51793722":"12:00:56"}
Thanks
BTW I looked at these two posts but don't understand them enough to implement.
Seems like what I am asking:
deserialize json
Not sure if this is even in the ball park:
Linq to XML query works when returning single entity but needs to be of list<T>
Warning:
this is an exercise to understand better JSON database design in Firebase
it is not necessarily realistic
I have got a two ways relationship between users and door keys. I would like to understand:
how to represent this relationship visually (I can imagine it only as two separate trees)
how this would work on Firebase, would both users and door-keys be child of a parent node "myparentnodename"?
If I model the database in this way it feels highly inefficient because every time I would query the child node "users" I would get all the users back. Or am I wrong? Is it possible to only get back the data matching to a specific user? E.g. get user where "user = user1"? Can we do nested queries? e.g. combine the previous condition with some condition on the door keys so the JSON object returned is only relevant to the door-keys contained in the "user1" node?
This is a very long answer as your question was actually about 5 different questions.
root node is: myparentnodename
Your users
users
uid_0
name: "William"
door_keys:
key_0: true
key_3: true
uid_2
name: "Leonard"
door_keys:
key_3: true
key_5: true
and your keys
keys
key_0
uid_0: true
key_3
uid_0: true
uid_2: true
key_5
uid_5: true
With this structure, all of the elements 'point' at each other.
If you query uid_0 you can see that they use keys 0 and 3
If you query key_3 you can see they belong to users 0 and 2
Your question was
every time I would query the child node "users" I would get all the
users back
That's slightly incomplete. When a query is done, you usually query for something specific. With Firebase however, there are two ways to retrieve data: observing a node and a query.
If you want back all users in the users node you would observe that node by .Value (or .ChildAdded for 1 at a time).
ref = myParentNodeName
let usersRef = myParentNodeName.childByAppendingPath("users")
usersRef.observeEventType(.Value, withBlock: { snapshot in
//.Value can return multiple nodes within the snapshot so iterate over them
for child in snapshot.children {
let name = child.value.objectForKey("name") as! String
print(name) //prints each users name
}
})
note that the above attaches an observer to the users node so any future changes within that node will notify the app and re-send the entire node to the app
If you want just one user's info, and don't want to continue watching for changes
ref = myParentNodeName
let usersRef = myParentNodeName.childByAppendingPath("users")
let thisUserRef = usersRef.childByAppendingPath("uid_2")
thisUserRef.observeSingleEventOfType(.Value, withBlock: { snapshot in
let name = child.value.objectForKey("name") as! String
print(name) //prints each users name
})
Finally, to query for all keys that belong to uid_0 (which is a little redundant in this example since we already know which keys they have from their node). If the keys ref also contained other info like the door name, the building the door was in, or the door location, it would be more appropriate and would require a different structure, so assume that's the case:
ref = myParentNodeName
let keysRef = myParentNodeName.childByAppendingPath("keys")
keysRef.queryOrderedByChild("uid_0").queryEqualToValue(true)
.observeSingleEventOfType(.Value, withBlock: { snapshot in
let doorLocation = child.value.objectForKey("door_location") as! String
print(doorLocation) //prints each users name
})
note this code is Swift since the platform was not specified in the question.
The other question:
Can we do nested queries? e.g. combine the previous condition with
some condition on the door keys so the JSON object returned is only
relevant to the door-keys contained in the "user1" node?
I think you mean can you query for uid_2, see which keys they have and then load in the info from those specific keys.
Yes! But... (there's always a but)
Firebase is asynchronous so you have to take that into account when nesting queries i.e. you need to ensure all of the data is returned before getting more data. So for example, if you wanted uid_2 key data, you could observeSingleEventOfType on node uid_2. You would then have their keys and could then observeSingleEventOfType on each key.
Technically this will work but with asynchronous data flying around, you could end up with code stomping on other code and processing data before it's actually been returned.
The better option (per my example) is to just avoid that entirely and query the keys node for uid_2's keys.
As a side note, observing a node has a lot less overhead than a query so if you want to load a single node and you know the path, use observe.
I have a set of data for a family tree in Neo4J and am trying to build a Cypher query that produces a JSON data set similar to the following:
{Name: "Bob",
parents: [
{Name: "Roger",
parents: [
Name: "Robert",
Name: "Jessica"
]},
{Name: "Susan",
parents: [
Name: "George",
Name: "Susan"
]}
]}
My graph has a relationship of PARENT between MEMBER nodes (i.e. MATCH (p.Member)-[:PARENT]->(c.Member) ). I found Nested has_many relationships in cypher and neo4j cypher nested collect which ends up grouping all parents together for the main child node I am searching for.
Adding some clarity based on feedback:
Every member has a unique identifier. The unions are currently all associated with the PARENT relationship. Everything is indexed so that performance will not suffer. When I run a query to just get back the node graph I get the results I expect. I'm trying to return an output that I can use for visualization purposes with D3. Ideally this will be done with a Cypher query as I'm using the API to access neo4j from the frontend being built.
Adding a sample query:
MATCH (p:Person)-[:PARENT*1..5]->(c:Person)
WHERE c.FirstName = 'Bob'
RETURN p.FirstName, c.FirstName
This query returns a list of each parent for five generations, but instead of showing the hierarchy, it's listing 'Bob' as the child for each relationship. Is there a Cypher query that would show each relationship in the data at least? I can format it as I need to from there...
Genealogical data might comply with the GEDCOM standard and include two types of nodes: Person and Union. The Person node has its identifier and the usual demographic facts. The Union nodes have a union_id and the facts about the union. In GEDCOM, Family is a third element bringing these two together. But in Neo4j, I found it suitable to also include the union_id in Person nodes. I used 5 relationships: father, mother, husband, wife and child. The family is then two parents with an inward vector and each child with an outward vector. The image illustrates this. This is very handy for visualizing connections and generating hypotheses. For example, consider the attached picture and my ancestor Edward G Campbell, the product of union 1917 where three brothers married three Vaught sisters from union 8944 and two married Gaither sisters from union 2945. Also, in the upper left, how Mahala Campbell married her step-brother John Greer Armstrong. Next to Mahala is an Elizabeth Campbell who is connected by marriage to other Campbell, but is likely directly related to them. Similarly, you can hypothesize about Rachael Jacobs in the upper right and how she might relate to the other Jacobs.
I use bulk inserts which can populate ~30000 Person nodes and ~100,000 relationships in just over a minute. I have a small .NET function that returns the JSon from a dataview; this generic solution works with any dataview so it is scalable. I'm now working on adding other data, such as locations (lat/long), documentation (particularly that linking folks, such as a census), etc.
You might also have a look at Rik van Bruggens Blog on his family data:
Regarding your query
You already create a path pattern here: (p:Person)-[:PARENT*1..5]->(c:Person) you can assign it to a variable tree and then operate on that variable, e.g. returning the tree, or nodes(tree) or rels(tree) or operate on that collection in other ways:
MATCH tree = (p:Person)-[:PARENT*1..5]->(c:Person)
WHERE c.FirstName = 'Bob'
RETURN nodes(tree), rels(tree), tree, length(tree),
[n in nodes(tree) | n.FirstName] as names
See also the cypher reference card: http://neo4j.com/docs/stable/cypher-refcard and the online training http://neo4j.com/online-training to learn more about Cypher.
Don't forget to
create index on :Person(FirstName);
I'd suggest building a method to flatten out your data into an array. If they objects don't have UUIDs you would probably want to give them IDs as you flatten and then have a parent_id key for each record.
You can then run it as a set of cypher queries (either making multiple requests to the query REST API, or using the batch REST API) or alternatively dump the data to CSV and use cypher's LOAD CSV command to load the objects.
An example cypher command with params would be:
CREATE (:Member {uuid: {uuid}, name: {name}}
And then running through the list again with the parent and child IDs:
MATCH (m1:Member {uuid: {uuid1}}), (m2:Member {uuid: {uuid2}})
CREATE m1<-[:PARENT]-m2
Make sure to have an index on the ID for members!
The only way I have found thus far to get the data I am looking for is to actually return the relationship information, like so:
MATCH ft = (person {firstName: 'Bob'})<-[:PARENT]-(p:Person)
RETURN EXTRACT(n in nodes(ft) | {firstName: n.firstName}) as parentage
ORDER BY length(ft);
Which will return a dataset I am then able to morph:
["Bob", "Roger"]
["Bob", "Susan"]
["Bob", "Roger", "Robert"]
["Bob", "Susan", "George"]
["Bob", "Roger", "Jessica"]
["Bob", "Susan", "Susan"]
Using sqlalchemy 0.7.2
Is there a way to find the table class from the query object? For example:
q = session.query(Customers)
how can I find Customers in q? Possible? Not Possible?
Yes. You need column_descriptions.
It's a long road to the table, though. sqlalchemy.orm.Query.column_descriptions returns a list of dicts, describing each query entity as it was given to query. in your example, there's only one entity, so you need the first item from that list. And since you're interested in the type of the query entity, rather than its' structure, you want the "type" key from that list:
q_entity = q.column_descriptions[0]['type']
assert q_entity == Customer
Accessing the table for the mapped class requires snooping around in the mapper subsystem. for that, you should use manager_of_class. The table is accessible from the manager through the mapper.mapped_table attribute:
from sqlalchemy.orm.attribute import manager_of_class
q_table = manager_of_class(q_entity).mapper.mapped_table
Resist the urge to skip strait to the mapper through Customer.__mapper__, or even Customer.__table__; That's specific to sqlalchemy.ext.declarative, and won't work with classes that are mapped by other means.