I have a SqlAlchemy model defined
from sqlalchemy.dialects.postgresql import JSONB
class User(db.Model):
__tablename__ = "user"
id = db.Column(db.Integer, primary_key=True)
nickname = db.Column(db.String(255), nullable=False)
city = db.Column(db.String(255))
contact_list = db.Column(JSONB)
created_at = db.Column(db.DateTime, default=datetime.utcnow)
def add_user():
user = User(nickname="Mike")
user.contact_list = [{"name": "Sam", "phone": ["123456", "654321"]},
{"name": "John", "phone": ["159753"]},
{"name": "Joe", "phone": ["147889", "98741"]}]
db.session.add(user)
db.session.commit()
if __name__ == "__main__":
add_user()
How can I retrieve the name from my contact_list using phone? For example, I have the 147889, how can I retrieve Joe?
I have tried this
User.query.filter(User.contact_list.contains({"phone": ["147889"]})).all()
But, it returns me an empty list, []
How can I do this?
You just forgot that your JSON path should include the outermost array as well:
User.query.filter(User.contact_list.contains([{"phone": ["147889"]}])).all()
will return the user you are looking for. The original query would match, if your JSON contained an object with key "phone" etc. Note that this returns the User object in question, not the specific object/name from the JSON structure. If you want that, as seems to be the end goal, you could expand the array elements of each user, filter based on the resulting records, and select the name:
val = db.column('value', type_=JSONB)
db.session.query(val['name'].astext).\
select_from(User,
db.func.jsonb_array_elements(User.contact_list).alias()).\
filter(val.contains({"phone": ["147889"]})).\
all()
On the other hand the above query is not as index friendly as the first one can be, because it has to expand all the arrays before filtering, so it might be beneficial to first find the users that contain the phone in their contact list in a subquery or CTE, and then expand and filter.
Related
I am new to django and I am working on project where I am storing a key value pair dictionary as JSON in database.
Now later I want to show it on the html page all the list of packages but not able to access those key value pair as dictionary.
here is my
models.py
class Packages(models.Model):
package_ID = models.AutoField("Package ID", primary_key=True)
package_Name = models.CharField("Package Name", max_length=30, null=False)
attribute_values = models.CharField("Item Details JSON", max_length=500, null=False)
package_Price = models.IntegerField("Package Price", null=False, default=0.00)
quantity = models.IntegerField("Quantity", null=False, default=00)
prod_ID = models.ForeignKey(Product, on_delete=models.CASCADE, verbose_name="Product ID (FK)")
its data entry is something like this
package_ID = 1
package_Name = "basic card"
attribute_values = {"sizes": "8.5 in. x 11 in.", "Colour": "Full-Color Front - Unprinted Back",}
package_Price = 200
quantity = 400
prod_ID = 1
package_ID = 2
package_Name = "best card"
attribute_values = {"sizes": "8.5 in. x 11 in.", "Colour": "Full-Color Front - Unprinted Back",}
package_Price = 200
quantity = 500
prod_ID = 1
here the problem is that attribute_values fields stores a JSON string so I have to convert it in dictionary first and then have to access its dictionary.
the steps I want to do:
get all the packages having same product key:
now get the attribute_values of those packages
and convert that JSON into dictionary
then display the attribute_values of each package separately in html in particular block.
I want output something like this:
but I am getting this:
With Django version >=3.1 one can simply use JSONField for storing JSON in databases.
It is supported for the following databases:
JSONField is supported on MariaDB 10.2.7+, MySQL 5.7.8+, Oracle,
PostgreSQL, and SQLite 3.9.0+ (with the JSON1 extension
enabled).
Using the JSONField should be pretty simple:
class Packages(models.Model):
# other fields
attribute_values = models.JSONField()
Let's say we have an instance of Packages we can simply set a dictionary / variable that can be valid JSON on to attribute_values:
package.attribute_values = {"sizes": "8.5 in. x 11 in.", "Colour": "Full-Color Front - Unprinted Back",}
package.save()
This will be stored as JSON in the database.
When we want to access this in python it would automatically be deserialized to their python value:
package.attribute_values['new_key'] = 'new value'
package.save()
Considering your current problem with the text field you need to simply convert your string to equivalent python objects. This can be done using json.loads:
import json
attribute_values_dictionary = json.loads(package.attribute_values)
You can also add a method to your model to do this for you:
import json
class Packages(models.Model):
# Your fields
def attribute_values_to_python(self):
return json.loads(self.attribute_values)
Now in the view you can simply write:
attribute_values_dictionary = package.attribute_values_to_python()
In the template you can simply write:
{{ package.attribute_values_to_python }}
It is possible to store json in postgres using the json data type. Check this tutorial for an introduction: http://www.postgresqltutorial.com/postgresql-json/
Consider I am storing the following json in such a field:
{
"address": {
"street1": "123 seasame st"
}
}
I want a to store separately a reference to the street field. For example, I might have another object which is using data from this json structure and wants to store a reference to where it got the data. Maybe something like this:
class Product():
__tablename__ = 'Address'
street_1 = Column(String)
data_source = ?
Now I could make data_source a string and just store namespaces like address.street, but if I did this postgres has no idea what that means. Working with that in queries would mean parsing the string and other inefficient stuff. Does postgres support referring to fields stored inside json data structures?
This question is related to JSON foreign keys in PostgreSQL , but in this case I don't necessarily want a fk relationship. I just want to create a reference, which is not necessarily enforced in the way a fk is.
update:
To be more clear, I want to reference the location of something in the json structure on another attribute and store that reference in a column. In the below code, Address.data_source is a reference to the location of the street data (for example address.street1 in this case)
class Address():
__tablename__ = 'Address'
street_1 = Column(String)
sample_id = Column(Integer, ForeignKey('DataSample.uid'))
data_source = ?
class DataSample():
__tablename__ = 'DataSample'
uid = Column(Integer, primary_key=True)
data = Column(JSONB)
body = {
"address": {
"street1": "123 seasame st"
}
}
datasample = DataSample(data=body)
address = Address(street_1=datasample.data['address']['street_1'],
sample_id=datasample.uid,
data_source=?)
As clarified, the question is seeking a way to flexibly specify a path within a JSON object of a particular record. Keys are being handled in normal columns. Constraints on JSONB fields are not available, and there is no specific support for specifying paths within JSON objects.
I worked with the following in SQL Fiddle using PostgreSQL 9.6:
CREATE TABLE datasample (
id integer PRIMARY KEY,
data jsonb
);
CREATE TABLE address (
id integer PRIMARY KEY,
street_1 text,
sample_id integer REFERENCES datasample (id),
data_source text
);
INSERT INTO datasample(id, data)
VALUES (1, '{"address":{"street_1": "123 seasame st"}}');
INSERT INTO address(id,street_1, sample_id, data_source)
VALUES (1,'123 seasame st',1,'datasample.data->''address''->>''street''');
A typical lookup of the street address (needed to retrieve street_1) would resemble:
SELECT datasample.data->'address'->>'street_1'
FROM datasample
WHERE id=1;
There is no special postgres type for identifying columns. Strings are the closest available and you will need to retrieve the string (or array of strings, or object containing strings, if one of those simplifies parsing) and use it to build the query. In tbe first code block, I stored it as the (escaped) fragment of query - 'datasample.data->''address''->>''street'''. Though longer, it would require only retrieval and unescaping to use in a new custom query. I did not find a way to use the string as a fragment within the same SQL statement, though it might be possible to combine it with other bits of text to form a full statement that could be run through EXECUTE.
I have a JSON list which captures one to many relationships.
For example, School can have multiple Class objects and Class can have multiple Student objects, but Student only belongs to one Class and one School:
{
"School": [ {
"id": 1,
"name": "Grad School",
"Class": [ {
"name": 101,
"Student": [ {
"name": 501,
"propertyA": "test"
}]
}]
}]
}
I am trying to convert this JSON example into an appropriate schema but the nesting is causing issues. Apollo appears to be able to help but the example below isn't very descriptive:
https://launchpad.graphql.com/4nqqqmr19
I'm looking for suggestions on how to handle this situation, whether that be through a JSON schema converter (which handles nested situations) or other.
I think you issue is not really the schema, which to me looks straightforward:
You have these types (everything dummy code as you have not specified in what language/framework you want to provide the GraphQL-Api):
SchoolType
id ID
name String
classes [Class]
students [Students]
ClassType
id ID
name String
school School
students [Student]
StudentType
id ID
name String
class Class
school School
Then we need an entry point
classQueryType
name "school"
argument :id, ID
resolve do
schools.where(id: argument["id"])
So we have the schema. The bigger work is probably to get the different types to access the JSON Schema in a way that the types above work.
So let's say, we read the JSON data somehow, with the structure you have.
const DATA = JSON.parse("your-example.json")
We need to convert this into different collections of objects, so we can query them dynamically:
schools = []
classes = []
people = []
def build_schools(data)
data.schools.for_each do |school|
schools.push(
name: school.name,
id: school.id,
classes: build_classes(school)
)
end
end
def build_classes(school)
ids = []
school.classes.for_each do |class|
ids.push(class.id)
classes.push(
id: class.id
name: class.name
school_id: school.id # you create your own references, to associate these objects
students: build_students(class)
)
end
return ids
end
...
But then you still need to hook this up, with your type system. Which means to write your resolvers:
For example on the StudentType
StudentType
id ID
name String
class Class
school School
resolve(object) ->
school_id = students.where(id: object.id).class_id.school_id
schools.where(id: school_id)
I have a function which returns json data as history from Version of reversion.models.
from django.http import HttpResponse
from reversion.models import Version
from django.contrib.admin.models import LogEntry
import json
def history_list(request):
history_list = Version.objects.all().order_by('-revision__date_created')
data = []
for i in history_list:
data.append({
'date_time': str(i.revision.date_created),
'user': str(i.revision.user),
'object': i.object_repr,
'field': i.revision.comment.split(' ')[-1],
'new_value_field': str(i.field_dict),
'type': i.content_type.name,
'comment': i.revision.comment
})
data_ser = json.dumps(data)
return HttpResponse(data_ser, content_type="application/json")
When I run the above snippet I get the output json as
[{"type": "fruits", "field": "colour", "object": "anyobject", "user": "anyuser", "new_value_field": "{'price': $23, 'weight': 2kgs, 'colour': 'red'}", "comment": "Changed colour."}]
From the function above,
'comment': i.revision.comment
returns json as "comment": "changed colour" and colour is the field which I have written in the function to retrieve it from comment as
'field': i.revision.comment.split(' ')[-1]
But i assume getting fieldname and value from field_dict is a better approach
Problem: from the above json list I would like to filter new_field_value and old_value. In the new_filed_value only value of colour.
Getting the changed fields isn't as easy as checking the comment, as this can be overridden.
Django-reversion just takes care of storing each version, not comparing.
Your best option is to look at the django-reversion-compare module and its admin.py code.
The majority of the code in there is designed to produce a neat side-by-side HTML diff page, but the code should be able to be re-purposed to generate a list of changed fields per object (as there can be more than one changed field per version).
The code should* include a view independent way to get the changed fields at some point, but this should get you started:
from reversion_compare.admin import CompareObjects
from reversion.revisions import default_revision_manager
def changed_fields(obj, version1, version2):
"""
Create a generic html diff from the obj between version1 and version2:
A diff of every changes field values.
This method should be overwritten, to create a nice diff view
coordinated with the model.
"""
diff = []
# Create a list of all normal fields and append many-to-many fields
fields = [field for field in obj._meta.fields]
concrete_model = obj._meta.concrete_model
fields += concrete_model._meta.many_to_many
# This gathers the related reverse ForeignKey fields, so we can do ManyToOne compares
reverse_fields = []
# From: http://stackoverflow.com/questions/19512187/django-list-all-reverse-relations-of-a-model
changed_fields = []
for field_name in obj._meta.get_all_field_names():
f = getattr(
obj._meta.get_field_by_name(field_name)[0],
'field',
None
)
if isinstance(f, models.ForeignKey) and f not in fields:
reverse_fields.append(f.rel)
fields += reverse_fields
for field in fields:
try:
field_name = field.name
except:
# is a reverse FK field
field_name = field.field_name
is_reversed = field in reverse_fields
obj_compare = CompareObjects(field, field_name, obj, version1, version2, default_revision_manager, is_reversed)
if obj_compare.changed():
changed_fields.append(field)
return changed_fields
This can then be called like so:
changed_fields(MyModel,history_list_item1, history_list_item2)
Where history_list_item1 and history_list_item2 correspond to various actual Version items.
*: Said as a contributor, I'll get right on it.
I'm using the sqlalchemy to store a binary tree data in the db:
class Distributor(Base):
__tablename__ = "distributors"
id = Column(Integer, primary_key=True)
upline_id = Column(Integer, ForeignKey('distributors.id'))
left_id = Column(Integer, ForeignKey('distributors.id'))
right_id = Column(Integer, ForeignKey('distributors.id'))
how can I generate json "tree" format data like the above listed:
{'id':1,children:[{'id':2, children:[{'id':3, 'id':4}]}]}
I'm guessing you're asking to store the data in a JSON format? Or are you trying to construct JSON from the standard relational data?
If the former, why don't you just create entries like:
{id: XX, parentId: XX, left: XX, right: XX, value: "foo"}
For each of the nodes, and then reconstruct the tree manually from the entries? Just start form the head (parentId == null) and then assemble the branches.
You could also add an additional identifier for the tree itself, in case you have multiple trees in the database. Then you would just query where the treeId was XXX, and then construct the tree from the entries.
I hesitate to provide this answer, because I'm not sure I really understand your the problem you're trying to solve (A binary tree, JSON, sqlalchemy, none of these are problems).
What you can do with this kind of structure is to iterate over each row, adding edges as you go along. You'll start with what is basically a cache of objects; which will eventually become the tree you need.
import collections
idmap = collections.defaultdict(dict)
for distributor in session.query(Distributor):
dist_dict = idmap[distributor.id]
dist_dict['id'] = distributor.id
dist_dict.setdefault('children', [])
if distributor.left_id:
dist_dict.['children'].append(idmap[distributor.left_id])
if distributor.right_id:
dist_dict.['children'].append(idmap[distributor.right_id])
So we've got a big collection of linked up dicts that can represent the tree. We don't know which one is the root, though;
root_dist = session.query(Distributor).filter(Distributor.upline_id == None).one()
json_data = json.dumps(idmap[root_dist.id])