I'm trying to make bulk data downloads by serializing my entire database as JSON. The drf documentation on serializers has a section that says you can simply do:
from rest_framework.renderers import JSONRenderer
serializer = CommentSerializer(comment)
json = JSONRenderer().render(serializer.data)
Unfortunately, this doesn't work for HyperLinked relationships. When you try to do it with them, you get something like:
AssertionError: HyperlinkedIdentityField requires the request in the serializer context. Add context={'request': request} when instantiating the serializer.
So, I figured out I can add context attribute, like:
r = Request(request=HttpRequest())
context = dict(request=r)
serializer = CommentSerializer(comment, context=context)
json = JSONRenderer().render(serializer.data)
Which then returns the error:
django.core.exceptions.ImproperlyConfigured: Could not resolve URL for hyperlinked relationship using view name "opinioncluster-detail". You may have failed to include the related model in your API, or incorrectly configured the lookup_field attribute on this field.
I know this API works properly when it's called from the browser, but I can't get past this when I call it as above. Something that's automatic from the browser doesn't happen when you render it manually.
Any ideas?
First edit
Here's another strategy that seemed promising because it would add the path to my request object:
r = Request(request=RequestFactory().get(reverse('comment-list', kwargs={'version': 'v3'})))
context = dict(request=r)
serializer = CommentSerializer(comment, context=context)
json = JSONRenderer().render(serializer.data)
That returns the same problem as if I hadn't defined a path to the request:
django.core.exceptions.ImproperlyConfigured: Could not resolve URL for hyperlinked relationship using view name "opinioncluster-detail". You may have failed to include the related model in your API, or incorrectly configured the lookup_field attribute on this field.
Second Edit
As I said in my comments, I'm fairly certain my serializers and views aren't to blame, since they work perfectly fine via the browser. Nevertheless, here they are. If you're truly generous, the full serializers, filters, and codebase is online.
View:
class OpinionClusterViewSet(LoggingMixin, viewsets.ModelViewSet):
queryset = OpinionCluster.objects.all()
serializer_class = OpinionClusterSerializer
filter_class = OpinionClusterFilter
ordering_fields = (
'date_created', 'date_modified', 'date_filed', 'citation_count',
'date_blocked',
)
Serializer:
class OpinionClusterSerializer(DynamicFieldsModelSerializer,
serializers.HyperlinkedModelSerializer):
absolute_url = serializers.CharField(source='get_absolute_url',
read_only=True)
panel = serializers.HyperlinkedRelatedField(
many=True,
view_name='judge-detail',
read_only=True,
)
non_participating_judges = serializers.HyperlinkedRelatedField(
many=True,
view_name='judge-detail',
read_only=True,
)
docket = serializers.HyperlinkedRelatedField(
many=False,
view_name='docket-detail',
read_only=True,
)
sub_opinions = serializers.HyperlinkedRelatedField(
many=True,
view_name='opinion-detail',
read_only=True,
)
class Meta:
model = OpinionCluster
This management command creates a MockRequest to be used in a non-browser environment and should allow you to create your JSON:
from django.core.management.base import BaseCommand
from django.http import HttpRequest
from rest_framework.renderers import JSONRenderer
from rest_framework.request import Request
from comments.models import Comment
from comments.serializers import CommentSerializer
class MockRequest(HttpRequest):
def __init__(self):
super(MockRequest, self).__init__()
self.setup_host()
def setup_host(self):
# Required to give absolute urls in output
self.META['HTTP_HOST'] = 'localhost:8000'
class Command(BaseCommand):
help = 'Export JSON'
def handle(self, *args, **options):
request = MockRequest()
serializer_context = {
'request': Request(request),
}
comment = Comment.objects.first()
serializer = CommentSerializer(comment, context=serializer_context)
json = JSONRenderer().render(serializer.data)
print(json)
To remove the hard coded domain you could use the Sites framework to power the host name:
def setup_host(self):
from django.contrib.sites.models import Site
site = Site.objects.get_current()
self.META['HTTP_HOST'] = site.domain
After digging quite deeply into the code, I finally figured this out:
r = RequestFactory().request()
r.version = 'v3'
r.versioning_scheme = URLPathVersioning()
context = dict(request=r)
renderer = JSONRenderer()
json_str = renderer.render(
serializer(item, context=context).data,
accepted_media_type='application/json; indent=2',
)
This seems to work, but the HyperLinkRelated values in the JSON that is serialized have the server set to testserver. I could get around that by setting:
r.META['SERVER_NAME']
But I also need to set r.scheme to https, which doesn't seem to be possible (I get an error that r.scheme cannot be set).
I'm pretty close though, so this is going to have to serve as an answer for now.
Related
Wondering what I'm doing wrong here. I'm running DRF with a React frontend. Trying to get one serializer to GET and POST a user's selected stations.
The AJAX POST works great, but the get on a page's initial load no longer works, and Django tells me: TypeError: 'Station' object is not iterable.
From reading around I'm vaguely aware that I shouldn't be declaring station with many = True, as this is causing the exception to be thrown. But I can't find any other way to get it working, it mangles the JSON POST request if I remove this option and stops the data validating.
The create is using custom code and is working fine.
Should I be trying to get this to work seamlessly or should I just hack it/make a different serializer to do the POST? Am I doing this in a logical way or have I got it all back-to-front?
models.py
class Station(models.Model):
network = models.ForeignKey(Network, on_delete=models.CASCADE)
name = models.CharField(db_column='name', max_length=256) # Field name made lowercase.
latitude = models.FloatField()
longitude = models.FloatField()
users = models.ManyToManyField('Account', through='UserStations')
class Meta:
managed = True
def __str__(self):
return self.name
class UserStations(models.Model):
station = models.ForeignKey(Station, on_delete=models.CASCADE)
user = models.ForeignKey(Account, on_delete=models.CASCADE)
serializers.py
class UserStationListSerializer(serializers.ModelSerializer):
class Meta:
model = Station
# fields = ('id', 'name')
fields = '__all__'
class UserStationsSerializer(serializers.ModelSerializer):
def __init__(self, *args, **kwargs):
#many = kwargs.pop('many', True)
print args
super(UserStationsSerializer, self).__init__(*args, **kwargs)
station = UserStationListSerializer(many=True, required = False, read_only=False)
views.py
class UserStationList(generics.ListCreateAPIView):
queryset = UserStations.objects.all()
serializer_class = UserStationsSerializer
Urls.py
from django.conf.urls import url
from . import views
from .views import StationList, AccountViewSet
from rest_framework.routers import DefaultRouter
from rest_framework.authtoken.views import obtain_auth_token
router = DefaultRouter()
router.register(r'users', AccountViewSet)
urlpatterns = router.urls
urlpatterns += [
url(r'^$', views.StationList.as_view(), name='station-list'),
url(r'user-station-list/', views.UserStationList.as_view(), name='user-station-list'),
url(r'^obtain-auth-token/$', obtain_auth_token),
url(r'station-list/', views.StationList.as_view(), name='user-station-list'),
]
I have two models (Like and News). I am using django-rest-framework to make a web api out of it.
class Like(models.Model):
user = models.ForeignKey(User)
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey('content_type', 'object_id')
class News(models.Model):
user = models.ForeignKey(User)
title = models.CharField(max_length=150)
...
likes = GenericRelation(Like)
A Like object has its own user field to store who liked the News object. Now to check if a specific user exists in any of the likes of a News object, I am getting request.user from a SerializerMethodField.
class NewsSerializer(serializers.HyperlinkedModelSerializer):
user = UserSerializer()
likes_count = serializers.IntegerField(source='likes.count', read_only=True)
user_in_likes = serializers.SerializerMethodField()
class Meta:
model = News
fields = ('user', 'title', 'body', 'article_image', 'pub_date', 'likes_count', 'user_in_likes')
def get_user_in_likes(self, obj):
user = self.context['request'].user
what = obj.likes.filter(user=user).exists()
return what
When I go the /news/ url, I get the json objects including the user_in_likes to true/false for each news object.
However, I have another serialzer for different model which imports NewsSerializer class and other similar serializers:
class ActivityObjectRelatedField(serializers.RelatedField):
def to_representation(self, value):
if isinstance(value, User):
serializer = UserSerializer(value)
elif isinstance(value, Job):
serializer = JobSerializer(value)
elif isinstance(value, News):
serializer = NewsSerializer(value)
elif isinstance(value, Tender):
serializer = TenderSerializer(value)
else:
raise Exception('Unexpected type of tagged object')
return serializer.data
class ActivitySerializer(serializers.HyperlinkedModelSerializer):
actor = ActivityObjectRelatedField(read_only=True)
target = ActivityObjectRelatedField(read_only=True)
class Meta:
model = Activity
fields = ('url', 'actor', 'verb', 'target', 'pub_date')
Now if I visit /activities/, to get the activities objects I am getting an error:
KeyError at /activities/
'request'
And it points to the line of SerializerMethod of NewsSerialize class where self.context['request'].user is used.
Exception Location: /vagrant/myproject/news/serializers.py in get_user_in_likes, line 25
Again if I visit /news/ url, everything is fine and I get news objects. What am I missing here? Why is request not being recognized in the ActivitiesSerializer class? Please help me solve this problem. Thank you.
You are getting this error because you are not passing request in the context when instantiating the NewsSerializer class or other similar serializers in the to_representation() method.
You are manually instantiating the particular serializer class in to_representation() method. So, after instantiation, that particular serializer does not have access to ActivitySerializer's context thereby leading to the error.
You can pass the ActivitySerializer's context during instantiation of the relevant serializer in the to_representation() method.
class ActivityObjectRelatedField(serializers.RelatedField):
def to_representation(self, value):
if isinstance(value, User):
serializer = UserSerializer(value, context=self.context) # pass context
elif isinstance(value, Job):
serializer = JobSerializer(value, context=self.context) # pass context
elif isinstance(value, News):
serializer = NewsSerializer(value, context=self.context) # pass context
elif isinstance(value, Tender):
serializer = TenderSerializer(value, context=self.context) # pass context
else:
raise Exception('Unexpected type of tagged object')
return serializer.data
It seems like you don't populate the context dictionary of NewsSerializer with your request in the /activities/ view.
You probably use a class based view provided by Django REST Framework which populates this dictionary for you (see the get_serializer_context() method) and passes it to the Serializer instance. That's why it's automatically available to your serializer in your /news/ view.
In your /activities/ view, though, the context is passed to ActivitySerializer and isn't (automatically) propagated further from there. That's why there's no request key in your context dictionary of NewsSerializer. You would need to pass your request object to the NewsSerializer instance. To pass extra context to a Serializer you can add a context parameter containing a dictionary when instantiating (see the DRF documentation).
I have started to play around with the Django REST framework. So far I succeeded in creating a serializer for my object, creating the post view, post objects and return objects via Javascript's $.post(). So right now I have a proper conversion between my JSONs and Django model objects.
The problem is that I have an array of objects [A1, A2, ..., An]. Right now when I need to post such an array I do it object by object. Is there any possibility to post the whole array at once, and recover an array of objects inside my Django View? If so, what is the pattern to follow here? I guess I could define a new model which is an array of my current model, create a serializer for it, etc., but this does not seem too elegant.
Below are my view and serializer:
#serializers.py
class SearchRequestSerializer(serializers.ModelSerializer):
def create(self):
return SearchRequest(**self.validated_data)
class Meta:
model = SearchRequest
#views.py
#api_view(['POST'])
def post_calculation(request):
if request.method == 'POST':
#JSON to serializer object
serializer = SearchRequestSerializer(data=request.data, many=False)
if (serializer.is_valid() == False):
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
#create the Python object
search_request = serializer.create()
#-- I do some processing stuff with the search_request object here ---
#object back to JSON
serializer3 = SearchRequestSerializer(search_request, many=False)
return Response(serializer3.data)
There are two solutions to your problem:
The first solution is to override the .create() method of your view
By default, django rest framework assumes you are passing it a single object. To account for the possibility to pass it a list of objects you might rewrite it as follows:
def create(self, request, pk=None, company_pk=None, project_pk=None):
is_many = isinstance(request.data, list)
serializer = self.get_serializer(data=request.data, many=is_many)
serializer.is_valid(raise_exception=True)
self.perform_create(serializer)
headers = self.get_success_headers(serializer.data)
return Response(serializer.data, status=status.HTTP_201_CREATED, headers=headers)
Now your view will handle both single objects in POST data as well as a list of objects.
The second solution is to use a third party package
django-rest-framework-bulk provides the above functionality plus additional features (e.g. bulk update). You can check it out and decide whether it fits your needs.
Update: The solution for function based views
In order to get it to work with your function based view, the approach is similar:
#api_view(['POST'])
def post_calculation(request):
if request.method == 'POST':
is_many = isinstance(request.data, list)
# JSON to serializer object
serializer = SearchRequestSerializer(data=request.data, many=is_many)
if not serializer.is_valid():
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
serializer.save()
#-- do some processing stuff here ---
return Response(serializer.data)
Do not call .create() method directly, use .save() instead. Also, when using many=False on serializer, the created instance is available under serializer.instance. I am not sure how to obtain the list of created instances though. You can try the same serializer.instance. If it doesn't work, try to find how to get it.
I had this working great less than DRF 2.4, but the change to 2.4+ uses #detail_route.
When I do a GET to /api/parent/7/children I expect to get all the children that belong to Parent 7.
But I'm getting an empty array.
Here's my code:
class ParentViewSet(viewsets.ModelViewSet):
serializer_class = ParentSerializer
model = models.Parent
#detail_route()
def children(self, request, pk):
parent = self.get_object()
children = parent.children.all()
serializer = ChildrenSerializer(children)
return Response(serializer.data)
def get_queryset(self):
if self.request.user.is_superuser:
return models.Parent.objects.all()
else:
return models.Parent.objects.filter(user=self.request.user)
def pre_save(self, obj):
obj.user = self.request.user
When I go to the endpoint /api/parent/7/children in the API viewer the response I get is:
{
"detail": "Not found"
}
Any suggestions?
I like this better in principle, you can be much more declarative about what is happening. I know I can set methods on children so I don't have to do a #link and #action for the same resource. Just need to get past this hurdle.
Thanks!
I don't know if this is your issue or not, but you have a typo in the example as presented...
children = parent.children.all()
serializer = ChildrenSerializer(Children)
Should be:
children = parent.children.all()
serializer = ChildrenSerializer(children)
You're passing the Children class to be serialized, not the children instance.
I'm learning Backbone.js and Flask (and Flask-sqlalchemy). I chose Flask because I read that it plays well with Backbone implementing RESTful interfaces. I'm currently following a course that uses (more or less) this model:
class Tasks(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(80), unique=True)
completed = db.Column(db.Boolean, unique=False, default=False)
def __init__(self, title, completed):
self.title = title
self.completed = completed
def json_dump(self):
return dict(title=self.title, completed=self.completed)
def __repr__(self):
return '<Task %r>' % self.title
I had to add a json_dump method in order to send JSON to the browser. Otherwise, I would get errors like object is not JSON serializable, so my first question is:
Is there a better way to do serialization in Flask? It seems that some objects are serializable but others aren't, but in general, it's not as easy as I expected.
After a while, I ended up with the following views to take care of each type of request:
#app.route('/tasks')
def tasks():
tasks = Tasks.query.all()
serialized = json.dumps([c.json_dump() for c in tasks])
return serialized
#app.route('/tasks/<id>', methods=['GET'])
def get_task(id):
tasks = Tasks.query.get(int(id))
serialized = json.dumps(tasks.json_dump())
return serialized
#app.route('/tasks/<id>', methods=['PUT'])
def put_task(id):
task = Tasks.query.get(int(id))
task.title = request.json['title']
task.completed = request.json['completed']
db.session.add(task)
db.session.commit()
serialized = json.dumps(task.json_dump())
return serialized
#app.route('/tasks/<id>', methods=['DELETE'])
def delete_task(id):
task = Tasks.query.get(int(id))
db.session.delete(task)
db.session.commit()
serialized = json.dumps(task.json_dump())
return serialized
#app.route('/tasks', methods=['POST'])
def post_task():
task = Tasks(request.json['title'], request.json['completed'])
db.session.add(task)
db.session.commit()
serialized = json.dumps(task.json_dump())
return serialized
In my opinion, it seems a bit verbose. Again, what is the proper way to implement them? I have seen some extensions that offer RESTful interfaces in Flask but those look quite complex to me.
Thanks
I would use a module to do this, honestly. We've used Flask-Restless for some APIs, you might take a look at that:
https://flask-restless.readthedocs.org/en/latest/
However, if you want build your own, you can use SQLAlchemy's introspection to output your objects as key/value pairs.
http://docs.sqlalchemy.org/en/rel_0_7/core/schema.html#metadata-reflection
Something like this, although I always have to triple-check I got the syntax right, so take this as a guide more than working code.
#app.route('/tasks')
def tasks():
tasks = Tasks.query.all()
output = []
for task in tasks:
row = {}
for field in Tasks.__table__.c:
row[str(field)] = getattr(task, field, None)
output.append(row)
return jsonify(data=output)
I found this question which might help you more. I'm familiar with SQLAlchemy 0.7 and it looks like 0.8 added some nicer introspection techniques:
SQLAlchemy introspection
Flask provides jsonify function to do this. Check out its working here.
Your json_dump method is right though code can be made concise. See this code snippet
#app.route('/tasks')
def tasks():
tasks = Tasks.query.all()
return jsonify(data=[c.json_dump() for c in tasks])