I want to serialize my queryset, and I want it in a format as this view outputs:
class JSONListView(ListView):
queryset = Users.objects.all()
def get(self, request, *args, **kwargs):
return HttpResponse(json.dumps({'data': [['bar','foo','bar','foo'],['foo','bar','foo','bar']]}, indent=4), content_type='application/json')
I simply don't know how to output the queryset instead of the manual data in the example.
I've tried
json.dumps({"data": self.get_queryset()})
and
serializers.serialize("json", {'data': self.get_queryset()})
but it won't work. What am I doing wrong? Do I need to make a custom JSON Encoder?
You can use JsonResponse with values. Simple example:
from django.http import JsonResponse
def some_view(request):
data = list(SomeModel.objects.values()) # wrap in list(), because QuerySet is not JSON serializable
return JsonResponse(data, safe=False) # or JsonResponse({'data': data})
Or another approach with Django's built-in serializers:
from django.core import serializers
from django.http import HttpResponse
def some_view(request):
qs = SomeModel.objects.all()
qs_json = serializers.serialize('json', qs)
return HttpResponse(qs_json, content_type='application/json')
In this case result is slightly different (without indent by default):
[
{
"model": "some_app.some_model",
"pk": 1,
"fields": {
"name": "Elon",
"age": 48,
...
}
},
...
]
I have to say, it is good practice to use something like marshmallow to serialize queryset.
...and a few notes for better performance:
use pagination if your queryset is big;
use objects.values() to specify list of required fields to avoid serialization and sending to client unnecessary model's fields (you also can pass fields to serializers.serialize);
It didn't work, because QuerySets are not JSON serializable.
1) In case of json.dumps you have to explicitely convert your QuerySet to JSON serializable objects:
class Model(model.Model):
def as_dict(self):
return {
"id": self.id,
# other stuff
}
And the serialization:
dictionaries = [ obj.as_dict() for obj in self.get_queryset() ]
return HttpResponse(json.dumps({"data": dictionaries}), content_type='application/json')
2) In case of serializers. Serializers accept either JSON serializable object or QuerySet, but a dictionary containing a QuerySet is neither. Try this:
serializers.serialize("json", self.get_queryset())
Read more about it here:
https://docs.djangoproject.com/en/dev/topics/serialization/
For a efficient solution, you can use .values() function to get a list of dict objects and then dump it to json response by using i.e. JsonResponse (remember to set safe=False).
Once you have your desired queryset object, transform it to JSON response like this:
...
data = list(queryset.values())
return JsonResponse(data, safe=False)
You can specify field names in .values() function in order to return only wanted fields (the example above will return all model fields in json objects).
To return the queryset you retrieved with queryset = Users.objects.all(), you first need to serialize them.
Serialization is the process of converting one data structure to another. Using Class-Based Views, you could return JSON like this.
from django.core.serializers import serialize
from django.http import JsonResponse
from django.views.generic import View
class JSONListView(View):
def get(self, request, *args, **kwargs):
qs = User.objects.all()
data = serialize("json", qs)
return JsonResponse(data)
This will output a list of JSON. For more detail on how this works, check out my blog article How to return a JSON Response with Django. It goes into more detail on how you would go about this.
If the goal is to build an API that allow you to access your models in JSON format I recommend you to use the django-restframework that is an enormously popular package within the Django community to achieve this type of tasks.
Django Rest Framework Website
Github
It include useful features such as Pagination, Defining Serializers, Nested models/relations and more. Even if you only want to do minor Javascript tasks and Ajax calls I would still suggest you to build a proper API using the Django Rest Framework instead of manually defining the JSON response.
Another way to turn queryset into JSON, is appending necessary elements to an empty list with loop. It provides to design customizable JSON.
queryset = Users.objects.all()
output = []
for query in queryset:
output.append('id': query.id, 'name': query.name, etc...)
return JSONResponse(output, safe=False)
Try this:
class JSONListView(ListView):
queryset = Users.objects.all()
def get(self, request, *args, **kwargs):
data = {}
data["users"] = get_json_list(queryset)
return JSONResponse(data)
def get_json_list(query_set):
list_objects = []
for obj in query_set:
dict_obj = {}
for field in obj._meta.get_fields():
try:
if field.many_to_many:
dict_obj[field.name] = get_json_list(getattr(obj, field.name).all())
continue
dict_obj[field.name] = getattr(obj, field.name)
except AttributeError:
continue
list_objects.append(dict_obj)
return list_objects
from django.http import JsonResponse
def SomeFunction():
dict1 = {}
obj = list( Mymodel.objects.values() )
dict1['data']=obj
return JsonResponse(dict1)
Try this code for Django
Related
I would like to create a django rest api that receives a csv file and then response the file in a json format. How do I achieve this without using models and anything related with databases? I don't want to use a database because it's assumed that the csv files will have different structure every time
This is my first django project and I've seen lots of tutorials, even I did the first tutorial from django website but I can't understand how to do this task without the database.
Thanks!
since you have not tried anything on your own Here is how you can do it
views.py
from rest_franework.generics import CreateAPIView
class ReadCSVView(CreateAPIView):
# permission_classes = [IsAuthenticated]
serializer_class = ReadCSVSerializer
queryset = ''
def perform_create(self, serializer):
file = serializer.validated_data['file']
decoded_file = file.read().decode()
io_string = io.StringIO(decoded_file)
reader = csv.reader(io_string)
next(reader) # incase you want to skip first row else remove this
return reader
def create(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
final_data = []
for row in self.perform_create(serializer):
if row[0]=="jack":
#do your logic here
final_data.append(row)
return Response(final_data, status=status.HTTP_201_CREATED)
just create one serializers to read csv.
serializers.py
from rest_framework import serializers
class ReadCSVSerializer(serializers.Serializer):
file = serializers.FileField()
now go to your urls.py and call the view class this way
urlpatterns = [
path("read-csv",views.ReadCSVView.as_view(),name="csv")]
hope this clarifies your doubt
I'm trying to get this test API working, it is very simple and the model consist of only one class with 2 fields.
My viewset inherits from ModelViewset and I added(to make it more explicit, and make some changes in the future) the list and create function.
I could POST a few JSONs with succes, but when I try to post this one that has a 'nested json' it returns
"Not a valid string."
Here is the code and the JSON:
views.py:
class SessionViewset(viewsets.ModelViewSet):
""""
A simple viewset for listing or retrieving 'sessions'
"""
queryset = Session.objects.all()
serializer_class = SessionSerializer
def list(self, request):
queryset = Session.objects.all()
serializer = SessionSerializer(queryset, many=True)
return Response(serializer.data)
def create(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
self.perform_create(serializer)
headers = self.get_success_headers(serializer.data)
return Response(serializer.data, status=status.HTTP_201_CREATED, headers=headers)
JSON:
{
"meta":"aca va la metadata",
"reunion":{"juan":500,
"pedro":300,
"jose":1000}
}
and it returns:
{
"reunion": [
"Not a valid string."
]
}
screenshot:
models:
models
serializers:
serializers
Do you know if there is any limitation that I'm not aware of, or how can I solve this issue?
Thanks
Tried to POST that JSON, into the API endpoint, with said error response
#MS25 as I mentioned in the comment your model has the reunion as CharField and you are sending an object data type to it.
If you still want to send the same data to the reunion then try something along the lines:
{
"meta":"aca va la metadata",
"reunion":JSON.stringify({"juan":500,
"pedro":300,
"jose":1000})
}
I am not sure of the results maybe it will work maybe not.
i am trying to make the following view with return JsonResponse() at the end work correctly:
def get_data(request):
full_data = Fund.objects.all()
data = {
"test2": full_data.values('investment_strategy').annotate(sum=Sum('commitment')),
}
return JsonResponse(data)
However, I get an error message saying "Object of type QuerySet is not JSON serializable".
When I put the above Queryset in a view with return render() at the end:
def get_more_data(request):
full_data = Fund.objects.all()
data = {"test2": full_data.values('investment_strategy').annotate(sum=Sum('commitment'))}
return render (request, 'test.html', data)
I get the the following result: <QuerySet [{'investment_strategy': 'Buyout', 'sum': 29}, {'investment_strategy': 'Growth', 'sum': 13}, {'investment_strategy': 'Miscellaneous', 'sum': 14}, {'investment_strategy': 'Venture Capital', 'sum': 23}, {'investment_strategy': 'n/a', 'sum': 36}]>
So the queryset works fine, I just have no clue how to return the data in proper Json format (which I would need to use the data charts.js)
I looked through answers for similar questions such as:
TypeError: object is not JSON serializable in DJango 1.8 Python 3.4
Output Django queryset as JSON
etc.
but could not find a meaningful solution for my problem.
Any help would be much appreciated!
So I managed to find a solution, that worked for me - in case anyone else has the same problem. I changed my view to the following:
def get_data(request):
full_data = Fund.objects.all()
full_data_filtered = full_data.values('investment_strategy').annotate(sum=Sum('commitment'))
labels = []
values = []
for d in full_data_filtered:
labels.append(d['investment_strategy'])
values.append(d['sum'])
data = {
"labels": labels,
"values": values,
}
return JsonResponse(data)
So basically I iterate over the Queryset and assign the values I need to lists, which can be passed to JsonResponse. I don't know if this is the most elegant way to do this (sure not), but it works and I can render my data in charts.js
JsonResponse(list(data)) will evaluate the Queryset (actually perform the query to the database) and turn it into a list that can be passed to JsonResponse.
This works because you use values and annotate, so the list is a list of dictionaries containing serializable fields.
In the example you mentioned, it didn't work because the Queryset was just returning a list of model instances, so wrapping in list() isn't enough. If you hadn't added values, you'd have had a list of Fund instances which are not serializable.
The best way I found was to create a custom QuerySet and Manager, it's not a lot of code and it is reusable!
I began by creating the custom QuerySet:
# managers.py but you can do that in the models.py too
from django.db import models
class DictQuerySet(models.QuerySet):
def dict(self):
values = self.values()
result = {}
for value in values:
id = value['id']
result[id] = value # you can customize the content of your dictionary here
return result
Then I created a custom Manager, it is optional but i prefer this way.
# managers.py, also optional
class DictManager(models.Manager):
def get_queryset(self):
return DictQuerySet(self.model, using=self._db)
Change the default manager in your model:
# models.py
from .managers import DictManager
class Fund(models.Model):
# ...
objects = DictManager()
# ...
And now you can call the dict() method from the query
# views.py
def get_data(request):
full_data = Fund.objects.all().dict()
return JsonResponse(full_data)
The response will be the full_data as a dictionary of dictionaries and each key is the primary key of the corresponding object.
If you intend to keep the same format for your JSONs, then you can use the same custom manager for all your models.
The answer to this question is confusing me.
Multiple Models in Django Rest Framework?
the answer is to a question of sending multipule models in a response. I have the same use case.
the author of the answer has this:
def get(self, request, format=None, **kwargs):
cart = get_cart(request)
cart_serializer = CartSerializer(cart)
another_serializer = AnotherSerializer(another_object)
return Response({
'cart': cart_serializer.data,
'another': another_serializer.data,
'yet_another_field': 'yet another value',
})
but I am keeping with the documentation.
http://www.django-rest-framework.org/api-guide/serializers/#serializing-objects
EXAMPLE FROM DOCS
serializer = CommentSerializer(comment)
serializer.data
# {'email': 'leila#example.com', 'content': 'foo bar', 'created': '2016-01-27T15:17:10.375877'}
from rest_framework.renderers import JSONRenderer
json = JSONRenderer().render(serializer.data)
json
# b'{"email":"leila#example.com","content":"foo bar","created":"2016-01-27T15:17:10.375877"}'
so which one is it? Do I JSON or not JSON. This is what I currently have.
def get(self, request, format=None):
searchcityqueryset = SearchCity.objects.all()
neighborhoodqueryset = SearchNeighborhood.objects.all()
serializedsearchcity = SearchCitySerializer(searchcityqueryset)
serializedsearchneighborhood = SearchNeighborhoodSerializer(neighborhoodqueryset)
jsonsearchcity = JSONRenderer().render(serializedsearchcity.data)
jsonsearchneighborhood = JSONRenderer().render(serializedsearchneighborhood.data)
return Response({
'searchcity': jsonsearchcity,
'searchneighborhood': jsonsearchneighborhood,
})
You don't need to do it.
From the doc:
Unlike regular HttpResponse objects, you do not instantiate Response
objects with rendered content. Instead you pass in unrendered data,
which may consist of any Python primitives.
Also JSONRenderer is default renderer class, which will be used to render.
So you can simple do this:
return Response({
'cart': cart_serializer.data,
'another': another_serializer.data,
'yet_another_field': 'yet another value',
})
I'm working in Django 1.8 and having trouble finding the modern way to do this.
This is what I've got, based on Googling and this blog post:
results = PCT.objects.filter(code__startswith='a')
json_res = []
for result in results:
json_res.append(result.as_dict())
return HttpResponse(json.dumps(json_res), content_type='application/json')
However this gives me 'PCT' object has no attribute 'as_dict'.
Surely there must be a neater way by now?
I was wondering if it was possible to use JSONResponse but frustratingly, the docs give no example of how to use JSONRespose with a queryset, which must be the most common use case. I have tried this:
results = PCT.objects.filter(code__startswith='a')
return JsonResponse(results, safe=False)
This gives [<PCT: PCT object>, <PCT: PCT object>] is not JSON serializable.
Simplest solution without any additional framework:
results = PCT.objects.filter(code__startswith='a').values('id', 'name')
return JsonResponse({'results': list(results)})
returns {'results': [{'id': 1, 'name': 'foo'}, ...]}
or if you only need the values:
results = PCT.objects.filter(code__startswith='a').values_list('id', 'name')
return JsonResponse({'results': list(results)})
returns {'results': [[1, 'foo'], ...]}
use values() to return a querydict, and pass that to json.dumps
values = PCT.objects.filter(code__startswith='a').values()
return HttpResponse(json.dumps(values), content_type='application/json')
https://docs.djangoproject.com/en/1.8/ref/models/querysets/#values
Most of these answers are out of date. Here's what I use:
views.py (returns HTML)
from django.shortcuts import render
from django.core import serializers
def your_view(request):
data = serializers.serialize('json', YourModel.objects.all())
context = {"data":data}
return render(request, "your_view.html", context)
views.py (returns JSON)
from django.core import serializers
from django.http import HttpResponse
def your_view(request):
data = serializers.serialize('json', YourModel.objects.all())
return HttpResponse(data, content_type='application/json')
Take a look at Django's serialization framework. It allows not only the XML format, but also JSON and YAML.
The accepted answer, using JsonResponse, is nice and simple. However, it does not return complete objects.
An alternative is to use Django's serializers. Here's an example copied verbatim from the admin actions documentation:
...
response = HttpResponse(content_type="application/json")
serializers.serialize("json", queryset, stream=response)
return response
This is very similar to what happens in Django's JsonResponse, as can be seen in the source.
The main difference is that JsonResponse calls json.dumps() directly, and does not know how to handle querysets, whereas the example above uses serializers.serialize('json', ...), which does know how to handle querysets, and returns complete objects that can also be de-serialized later on.
If you want to save directly to file (using content-disposition: attachment to open a save dialog in the browser), you could use a FileResponse, for example:
...
data = serializers.serialize('json', queryset)
return FileResponse(
io.BytesIO(data.encode('utf-8')),
content_type='application/json',
as_attachment=True,
filename=f'{queryset.model.__name__.lower()}-objects.json'
)