I have class of classes as below:
class Base:
def __init__(self):
self.var1 = “variable”
class Child:
def __init__(self):
self.base = Base()
self.var2 = False
class RootChild:
def __init__(self):
self.child = Child()
self.var3 = “variable”
self.base = Base()
I want to save an instance of ‘RootChild’ with all fields of it represented in json. There are a lot of topics about this task, but I could not understand their differences and use cases. What are most pythonic solutions to this problem?
Your mileage will vary - from using a "serializer" strategy: a layer of mechanisms that will know how to map from your classes to Json and back,
which can be a lot of work (to the point of hand-describing each field that should be serialized/unserialized) - https://docs.pylonsproject.org/projects/colander/en/latest/
Or you could build a custom JSON encoder that will just check if it is a known class and then return a dict with all its fields (which can be extracted automatically by introspection) - (check the example to extend the encoder here: https://docs.python.org/3/library/json.html )
Any way there are lots of trade-offs and differing level of work -
if you plan to read the JSON back in Python apps which have the same classes, you could use "jsonpickle" - it will add extra meta-data to your JSON, but will ensure round tripping of all your objects exactly as they are: https://jsonpickle.github.io/
Without using any other lib, and assuming you just need to export your classes, the custom encoder strategy can be somewhat simple:
import json
class SimpleObject(json.JSONEncoder):
def default(self, obj):
if hasattr(obj, "__dict__"):
return {key:value for key, value in obj.__dict__.items() if not key.startswith("_")}
return super().default(obj)
a = RootChild()
And on the interactive environment:
In [52]: a = RootChild()
In [53]: print(json.dumps(a, cls=SimpleObject, indent=4))
{
"child": {
"base": {
"var1": "variable"
},
"var2": false
},
"var3": "variable",
"base": {
"var1": "variable"
}
}
Related
i am trying to make the following view with return JsonResponse() at the end work correctly:
def get_data(request):
full_data = Fund.objects.all()
data = {
"test2": full_data.values('investment_strategy').annotate(sum=Sum('commitment')),
}
return JsonResponse(data)
However, I get an error message saying "Object of type QuerySet is not JSON serializable".
When I put the above Queryset in a view with return render() at the end:
def get_more_data(request):
full_data = Fund.objects.all()
data = {"test2": full_data.values('investment_strategy').annotate(sum=Sum('commitment'))}
return render (request, 'test.html', data)
I get the the following result: <QuerySet [{'investment_strategy': 'Buyout', 'sum': 29}, {'investment_strategy': 'Growth', 'sum': 13}, {'investment_strategy': 'Miscellaneous', 'sum': 14}, {'investment_strategy': 'Venture Capital', 'sum': 23}, {'investment_strategy': 'n/a', 'sum': 36}]>
So the queryset works fine, I just have no clue how to return the data in proper Json format (which I would need to use the data charts.js)
I looked through answers for similar questions such as:
TypeError: object is not JSON serializable in DJango 1.8 Python 3.4
Output Django queryset as JSON
etc.
but could not find a meaningful solution for my problem.
Any help would be much appreciated!
So I managed to find a solution, that worked for me - in case anyone else has the same problem. I changed my view to the following:
def get_data(request):
full_data = Fund.objects.all()
full_data_filtered = full_data.values('investment_strategy').annotate(sum=Sum('commitment'))
labels = []
values = []
for d in full_data_filtered:
labels.append(d['investment_strategy'])
values.append(d['sum'])
data = {
"labels": labels,
"values": values,
}
return JsonResponse(data)
So basically I iterate over the Queryset and assign the values I need to lists, which can be passed to JsonResponse. I don't know if this is the most elegant way to do this (sure not), but it works and I can render my data in charts.js
JsonResponse(list(data)) will evaluate the Queryset (actually perform the query to the database) and turn it into a list that can be passed to JsonResponse.
This works because you use values and annotate, so the list is a list of dictionaries containing serializable fields.
In the example you mentioned, it didn't work because the Queryset was just returning a list of model instances, so wrapping in list() isn't enough. If you hadn't added values, you'd have had a list of Fund instances which are not serializable.
The best way I found was to create a custom QuerySet and Manager, it's not a lot of code and it is reusable!
I began by creating the custom QuerySet:
# managers.py but you can do that in the models.py too
from django.db import models
class DictQuerySet(models.QuerySet):
def dict(self):
values = self.values()
result = {}
for value in values:
id = value['id']
result[id] = value # you can customize the content of your dictionary here
return result
Then I created a custom Manager, it is optional but i prefer this way.
# managers.py, also optional
class DictManager(models.Manager):
def get_queryset(self):
return DictQuerySet(self.model, using=self._db)
Change the default manager in your model:
# models.py
from .managers import DictManager
class Fund(models.Model):
# ...
objects = DictManager()
# ...
And now you can call the dict() method from the query
# views.py
def get_data(request):
full_data = Fund.objects.all().dict()
return JsonResponse(full_data)
The response will be the full_data as a dictionary of dictionaries and each key is the primary key of the corresponding object.
If you intend to keep the same format for your JSONs, then you can use the same custom manager for all your models.
I have a backend server in python (Flask + Graphene) and I need to return a JSON object like this:
{
's1': "Section 1",
's2': "Section 2",
's3': "Section 3",
's4': "Section 4"
}
The resolver looks like below:
questionnaire = graphene.types.json.JSONString(
description='JSON result test')
def resolve_questionnaire(self, info: graphql.ResolveInfo):
sections = {
's1': "Section 1",
's2': "Section 2",
's3': "Section 3",
's4': "Section 4"
}
print(json.dumps(sections))
return sections
and in console I see as a result of print(json.dumps(sections)) as I expect:
user-api_1 | {"s1": "Section 1", "s2": "Section 2", "s3": "Section 3", "s4": "Section 4"}
But in GraphiQL i see all quotation marks with backslashes:
When I change the return sections to return json.dumps(sections) I get the result like this:
The question is how to properly return a JSON object in graphene resolver?
I know that there is json.replace method to use like here, but I believe that I am simply producing/passing object in wrong way.
Your initial result of
{
"data": {
"questionnaire": "{\"s1\": \"Section 1\", \"s2\": \"Section 2\", \"s3\": \"Section 3\", \"s4\": \"Section 4\"}"
}
}
is the intended behavior. After all, questionnaire resolves to a JSONString. Since it is a string it must be double quoted, thus its inner quotations must be escaped. This is according to JSON's standards.
To use that string you, would have to run some sort of JSON parser on the data.questionnaire object. In javascript, for instance, it would be something like:
var data;
// Fetching logic to get the data object from your GraphQL server
var sections = JSON.parse(data.questionaire);
// Now you can access its objects
console.log(sections.s1) // Should print "Section 1" on the dev console
However, the method described above is not ideal if the keys of sections are not predetermined (sections.s5 may be defined in one case but undefined in another). Instead, you might rather have an array that you can iterate over. To do this, you would have to define a "model" that has explicit key-value pairs. Doing this way is format suited for GraphQL, too. For instance:
import graphene
# Our new model
class Section(graphene.ObjectType):
key = graphene.String() # dictionary key
header = graphene.String() # dictionary value
# Your previous schema with modifications
class Query(graphene.ObjectType):
# questionnaire = graphene.types.json.JSONString(description='JSON result test')
# Return a list of section objects
questionnaire = graphene.List(Section)
def resolve_questionnaire(self, info: graphql.ResolveInfo):
sections = {
's1': "Section 1",
's2': "Section 2",
's3': "Section 3",
's4': "Section 4"
}
sections_as_obj_list = [] # Used to return a list of Section types
# Create a new Section object for each item and append it to list
for key, value in sections.items(): # Use sections.iteritems() in Python2
section = Section(key, value) # Creates a section object where key=key and header=value
sections_as_obj_list.append(section)
# return sections
return sections_as_obj_list
Now, if we run the query:
query {
questionnaire {
key
header
}
}
It returns a JSON array that can be iterated through.
{
"data" {
"questionnaire": [
{
"key": "s1",
"header": "Section 1"
},
{
"key": "s2",
"header": "Section 2"
},
{
"key": "s3",
"header": "Section 3"
},
{
"key": "s4",
"header": "Section 4"
},
]
}
}
Graphene now has a GenericScalar type for this.
from graphene.types import generic
...
errors = generic.GenericScalar()
You can just subclass the JSON type and replace the serialize method:
class Any(JSON):
#staticmethod
def serialize(dt):
return dt
Then instead of
questionnaire = Field(JSON)
write
questionnaire = Field(Any)
Yes, this does break the strictly-typed spirit of GraphQL, but if that's what you want to do, there's how to do it. Note that this is an output-only hack — it won't allow you to accept arbitrary structures as arguments.
In my case I have the JSON column called (details).
from graphene.types.scalars import Scalar
class ObjectField(Scalar):
''' convert the Json String into Json '''
#staticmethod
def serialize(dt):
return dt
#staticmethod
def parse_literal(node):
return node.value
#staticmethod
def parse_value(value):
return value
class CustomDiseaseFactNode(graphene.Node):
class Meta:
name = 'diseaseFactNode'
#staticmethod #this class used to get the primary key object id
def to_global_id(type, id):
return id
Call the JSONScalar from your object class
class DiseaseFactNode(SQLAlchemyObjectType):
"""DiseaseFact node."""
class Meta:
model = DiseaseFact
interfaces = (CustomDiseaseFactNode,)
details = JSONScalar()
Another approach, to return an array of json / dictionary.
I was trying to return json from a model mixin property attribute.
Basically an any type ( note: loses the benefits of typing ), helped me to return json from a list of dictionaries:
import graphene
from graphene import relay, Scalar
from graphene_django import DjangoObjectType
class DictType(Scalar):
#staticmethod
def serialize(dt):
return dt
#staticmethod
def parse_literal(node):
return node
#staticmethod
def parse_value(value):
return value
The node itself was based on a model which included a mixin:
class InvoiceFileNode(DjangoObjectType):
signed_url = graphene.String()
variable_files = graphene.List(of_type=DictType)
class Meta:
model = InvoiceFile
interfaces = (relay.Node,)
filter_fields = []
only_fields = (
"id",
"created",
"signed_url",
"variable_files",
)
def resolve_signed_url(self, *args, **kwargs):
# #property access from FileMixin
return self.signed_url
def resolve_openable_signed_url(self, *args, **kwargs):
# #property access from FileMixin
return self.openable_signed_url
The following mixin was what I was trying to get returned, but using the of_type of JSONString serialized the dictionary to a json string:
class FileMixin(object):
#property
def signed_url(self) -> str:
return get_signed_url(self.file)
#property
def variable_files(self) -> str:
sizes = []
for s in [128, 240, 720]:
item = {"width": s}
urls = []
for n in [1,2,3]:
urls.append(get_sized_url(self.file, n))
item["urls"] = urls
sizes.append(item)
return sizes
class InvoiceFile(Models, FileMixin):
created = DateTimeField()
file = CharField()
I was having issues with returning something like:
[{"width": 123, "stuff": [{"more": "stuff"}]}}
note
this will probably not work if the dictionary returned contains any sorts of functions, or objects, etc.
I have a Python 3.5 program that creates an inventory of objects. I created a class of Trampolines (color, size, spring, etc.). I constantly will create new instances of the class and I then save a dictionary of them. The dictionary looks like this:
my_dict = {name: instance} and the types are like so {"string": "object"}
My issue is that I want to know how to save this inventory list so that I can start where I left off the last time I closed the program.
I don't want to use pickle because I'm trying to learn secure ways to do this for more important versions in the future.
I thought about using sqlite3, so any tips on how to do this easily would be appreciated.
My preferred solution would state how to do it with the json module. I tried it, but the error I got was:
__main__.Trampoline object at 0x00032432... is not JSON serializable
Edit:
Below is the code I used when I got the error:
out_file = open(input("What do you want to save it as? "), "w")
json.dump(my_dict, out_file, indent=4)
out_file.close()
End of Edit
I've done a good amount of research, and saw that there's also an issue with many of these save options that you can only do one object per 'save file', but that the work around to this is that you use a dictionary of objects, such as the one I made. Any info clarifying this would be great, too!
What you might be able to do is saving the instance's attributes to a CSV-file and then just create it when starting up. This might be a bit too much code and is possible not the best way. One obvious problem is that it doesn't work if you don't have the same amount of attributes as parameters, which should be possible to fix if necessary I believe. I just thought I might try and post and see if it helps :)
import json
class Trampoline:
def __init__(self, color, size, height, spring):
self.color = color
self.size = size
self.height = height
self.spring = spring
def __repr__(self):
return "Attributes: {}, {}, {}, {}".format(self.color, self.size, self.height, self.spring)
my_dict = {
"name1": Trampoline('red', 100, 2.3, True),
"name2": Trampoline('blue', 50, 2.1, False),
"name3": Trampoline('green', 25, 1.8, True),
"name5": Trampoline('white', 10, 2.6, False),
"name6": Trampoline('black', 0, 1.4, True),
"name7": Trampoline('purple', -33, 3.0, True),
"name8": Trampoline('orange', -999, 2.5, False),
}
def save(my_dict):
with open('save_file.txt', 'w') as file:
temp = {}
for name, instance in my_dict.items():
attributes = {}
for attribute_name, attribute_value in instance.__dict__.items():
attributes[attribute_name] = attribute_value
temp[name] = attributes
json.dump(temp, file)
def load():
with open('save_file.txt', 'r') as file:
my_dict = {}
x = json.load(file)
for name, attributes in x.items():
my_dict[name] = Trampoline(**attributes)
return my_dict
# CHECK IF IT WORKS!
save(my_dict)
my_dict = load()
print("\n".join(["{} | {}".format(name, instance) for name, instance in sorted(my_dict.items())]))
Here is an example of a class that handles datetime objects.
class CustomEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, datetime.datetime):
if obj.tzinfo:
obj = obj.astimezone(isodate.tzinfo.UTC).replace(tzinfo=None)
return obj.isoformat()[:23] + 'Z'
return json.JSONEncoder.default(self, obj)
when you encode to json the default function of the cls is called with object you passed. If you want to handle a type that is not part of the standard json.JSONEncoder.default you need to intercept it and return how you want it handled as a valid json type. In this example I turned the datetime into a str and returned that. If its not one of the types I want to special case, I just pass it along to the standard json.JSONEncoder.default handler.
To use this class you need to pass it in the cls param of json.dump or json.dumps:
json.dumps(obj, cls=CustomEncoder)
Decoding is done the same way but with json.JSONDecoder, json.load, and json.loads. However you can not match on type, so you will need to either add an 'hint' in encoding for decoding or know what type it needs to decode.
For a simple class, you can make an easy serializer as below. This will take all of the properties of your Trampoline object and put them into a dictionary and then into JSON.
class Trampoline(object):
...
def serialize(self):
return json.dumps(vars(self))
If your class is a bit more complicated, then write a more complicated serializer :)
I am trying to implement a generic pattern with which to generate marshallers and unmarshallers for an Akka HTTP REST service using Argonaut, handling both entity and collection level requests and responses. I have no issues in implementing the entity level as such:
case class Foo(foo: String)
object Foo {
implicit val FooJsonCodec = CodecJson.derive[Foo]
implicit val EntityEncodeJson = FooJson.Encoder
implicit val EntityDecodeJson = FooJson.Decoder
}
I am running into issues attempting to provide encoders and decoders for the following:
[
{ "foo": "1" },
{ "foo": "2" }
]
I have attempted adding the following to my companion:
object Foo {
implicit val FooCollectionJsonCodec = CodecJson.derive[HashSet[Foo]]
}
However, I am receiving the following error:
Error:(33, 90) value jencode0L is not a member of object argonaut.EncodeJson
I see this method truly does not exist but is there any other generic method to generate my expected result. I'm strongly avoiding using an additional case class to describe the collection since I am using reflection heavily in my use case.
At this point, I'd even be fine with a manually constructed Encoder and Decoder, however, I've found no documentation on how to construct it with the expected structure.
Argonaut have predefined encoders and decoders for Scala's immutable lists, sets, streams and vectors. If your type is not supported explicitly, as in the case of java.util.HashSet, you can easily add EncodeJson and DecodeJson for the type:
import argonaut._, Argonaut._
import scala.collection.JavaConverters._
implicit def hashSetEncode[A](
implicit element: EncodeJson[A]
): EncodeJson[java.util.HashSet[A]] =
EncodeJson(set => EncodeJson.SetEncodeJson[A].apply(set.asScala.toSet))
implicit def hashSetDecode[A](
implicit element: DecodeJson[A]
): DecodeJson[java.util.HashSet[A]] =
DecodeJson(cursor => DecodeJson.SetDecodeJson[A]
.apply(cursor)
.map(set => new java.util.HashSet(set.asJava)))
// Usage:
val set = new java.util.HashSet[Int]
set.add(1)
set.add(3)
val jsonSet = set.asJson // [1, 3]
jsonSet.jdecode[java.util.HashSet[Int]] // DecodeResult(Right([1, 3]))
case class A(set: java.util.HashSet[Int])
implicit val codec = CodecJson.derive[A]
val a = A(set)
val jsonA = a.asJson // { "set": [1, 3] }
jsonA.jdecode[A] // DecodeResult(Right(A([1, 3])))
Sample is checked on Scala 2.12.1 and Argonaut 6.2-RC2, but as far as I know it shouldn't depend on some latest changes.
Approach like this works with any linear or unordered homogenous data structure that you want to represent as JSON array. Also, this is preferable to creating a CodecJson: latter can be inferred automatically from JsonEncode and JsonDecode, but not vice versa. This way, your set will serialize and deserialize both when used independently or within other data type, as shown in example.
I don't use Argonaut but use spray-json and suspect solution can be similar.
Have you tried something like this ?
implicit def HashSetJsonCodec[T : CodecJson] = CodecJson.derive[Set[T]]
if it doesn't work I'd probably try creating more verbose implicit function like
implicit def SetJsonCodec[T: CodecJson](implicit codec: CodecJson[T]): CodecJson[Set[T]] = {
CodecJson(
{
case value => JArray(value.map(codec.encode).toList)
},
c => c.as[JsonArray].flatMap {
case arr: Json.JsonArray =>
val items = arr.map(codec.Decoder.decodeJson)
items.find(_.isError) match {
case Some(error) => DecodeResult.fail[Set[T]](error.toString(), c.history)
case None => DecodeResult.ok[Set[T]](items.flatMap(_.value).toSet[T])
}
}
)
}
PS. I didn't test this but hopefully it leads you to the right direction :)
This question already has an answer here:
Rendering spatial data of GeoQuerySet in a custom view on GeoDjango
(1 answer)
Closed 2 years ago.
I am in the process of building a website on geodjango. On the front end I have a map on openlayers and I want to be able to fire ajax queries at some django url and get back geojson.
The problem is that the standard django json serializer doesn't do geojson. So I have a model like:
class Asset (models.Model):
objects = models.GeoManager()
url_name = models.CharField(max_length=200)
name = models.CharField(max_length=200)
point = models.PointField(srid=4326)
def __unicode__(self):
return self.name
And I want to create a view:
def geojson_query(request):
#geographic query
...
assets = Asset.objects.all().filter(point__contained=bb_4326)
json = serializers.serialize("json", assets)
return HttpResponse(json)
But then I get back a response that looks more like this:
"url_name": "Upper_Svaneti",
"name": "Upper Svaneti",
"point": "POINT (43.0113899999999987 42.9163899999999998)"
Whereas I want something that looks like this:
"url_name": "Upper_Svaneti",
"name": "Upper Svaneti",
"geometry": {
"type": "Point",
"coordinates": [43.0113899999999987 42.9163899999999998]
}
So I guess my question is, what is the best way to implement a geojson serializer in django without totally rolling my own? And if it comes to rolling my own, where do I start?
You need to write your own serializer. Just inherit from the DjangoJSONEncoder, here's one I created that supports the Point type:
from django.core.serializers.json import DjangoJSONEncoder
from django.contrib.gis.geos import Point
class GeoJSONEncoder(DjangoJSONEncoder):
def default(self, obj):
if isinstance(obj, Point):
return obj.coords
return super(GeoJSONEncoder, self).default(obj)
You can then use it like so:
from my_app.serializers import GeoJSONEncoder
from django.utils import simplejson
json = simplejson.dumps(data, cls=GeoJSONEncoder)
So... I have done something slightly unpretty. I hardcoded the non-geojson parts of the serializer and used the json function from GEOS to get the geojson part.
So the method in the model looks like:
def get_footprint_json(self):
geojson=self.footprint.json
json='{"type": "Feature","geometry": %s,"properties": {"name":"%s","url_name":"%s"}}'%(geojson,self.name,self.url_name)
return json
And... I have a view that looks like this:
json='{ "srid":4326, "type": "FeatureCollection","features": ['+','.join([asset.get_footprint_json() for asset in assets])+'] }'
return HttpResponse(json)
I'd be curious to see if anyone else has a better way or if django has updated their serializer to include geojson.