Im using json response in django, but I have special characters (ñáé etc...)
my view
def get_agencies(request):
qr = Agency.objects.all()
qr_jason = serializers.serialize('json',qr)
return JsonResponse(qr_jason, safe=False)
But if I enter a special character like ñ in the json I recieve the ascii equivalent. Actually I can make a dictionary and then make the JasonResponse with the dictionary and it works, I can't find a way to use the serializers.serialize with utf-8.
json recieved (the u00f1 are ñ)
// 20170124165944
// http://localhost:8080/get_agencies/
"[
{
\"model\": \"items.agency\",
\"pk\": 1,
\"fields\": {
\"name\": \"asdk\\u00f1ld\",
\"tipo\": \"librevile\",
\"adress\": \"laslkfdli323,
ls\\u00f1\\u00f1\",
\"phone\": \"56549875\",
\"web\": \"http: //www.systmatic.com.mx\",
\"lat\": 23.514646,
\"lng\": -26.152684,
\"created\": \"2017-01-24T00: 56: 28.302Z\",
\"last_updated\": \"2017-01-24T22: 22: 08.856Z\"
}
}
]"
Faster solution:
def get_agencies(request):
qr = Agency.objects.all().values()
qr_list = list(qr)
return JsonResponse(qr_list, , safe=False, json_dumps_params={'ensure_ascii':False})
I know that you wrote that you would like to serialize using django.core.serializers.serialize but... you could do a workaround and serialize using json standard lib.
import json
def get_agencies(request):
qr = Agency.objects.all().values()
qr_json = json.dumps(list(qr), ensure_ascii=False, default=str)
return JsonResponse(qr_json, safe=False)
I've added default=str parameter to json.dumps because I saw that you have a datetime field in your model, so that should take care of that issue.
Related
I am making a curl call to rest api visa curl in groovy. Response is coming fine but the response is very large, it is a 17MB of data, following is my script :
def converter = "curl.......'"
def initialSize = 4096
def out = new ByteArrayOutputStream(initialSize)
def err = new ByteArrayOutputStream(initialSize)
def process = [ 'bash', '-c', converter].execute()
process.consumeProcessOutput(out, err)
process.waitFor()
Curl response is coming fine, when I print response on console ,store in variable out, it gives response data where it is not neat json as I see some "/n" characters. When I write this to file then I dont see any new line and neat json, all I see data in one line in key value format.
{"key1":"value1","key2":"value2",} in one huge line only
This is when i view in sublime. Now I want to convert this to pretty json and write neatly into file.I tried following to approaches but both prints empty ({ }) in console and in file.
def json = JsonOutput.toJson(out)
println new JsonBuilder(out).toPrettyString()
What did I miss?
I am trying to use groovy libraries only.
UPDATE:
As i try to debug, i found that it may be because all JSON parsers expect string but my output is ByteArrayOutputStream. But now how can I convert the out to string ? I tried out.toString and out.text, it does not work.
Use StringWriter instead of ByteArrayOutputStream
Then JsonOutput.prettyPrint( stringWriter.toString() )
From a separate system I get a String parameter "messageJson" whose content is in the form:
{"agent1":"smith","agent2":"brown","agent3":{"agent3_1":"jones","agent3_2":"johnson"}}
To use it in my program I parse it with JsonSlurper.
def myJson = new JsonSlurper().parseText(messageJson)
But the resulting Json has the form:
[agent1:smith, agent2:brown, agent3:[agent3_1:jones, agent3_2:johnson]]
Note the square brackets and the lack of double quotes. How can I parse messageJson so that the original structure is kept?
Ok, thanks to the hint by cfrick, I was able to find a solution. In case anyone else has a similar problem, all I needed to do was using JsonOutput in the end to convert the map back to a Json
I.E. :
def myJson = new JsonSlurper().parseText(messageJson)
myJson << [agent4:"jane"]
def backToJson = JsonOutput.toJson(myJson)
I have a JSON file with some lines like:
"updatedAt" : ISODate("2018-11-20T09:32:16.732+0000"),
I tried json.loads but it has an error json.decoder.JSONDecodeError: Expecting value: line 2 column 13 (char 15).
I believe that the problem is at ISODate () but how could I handle that with Python?
Many thanks
This is not valid JSON, to begin with. I guess the ISODATE("...") is generated from MongoDB, maybe dumping the ISODate() helper directly instead of its string representation into the JSON?
In any case, you could use a regex on the whole JSON-string to get rid of the ISODate("..."), retrieve the date as a string and then use python-dateutil to parse the value to a datetime.datetime.
Something to the tune of
import json
import dateutil.parse
import re
json_str = ....
clean_json = re.compile('ISODate\(("[^"]+")\)').sub('\\1', json_str)
json_obj = json.loads(clean_json)
# use dateutil.parser.parse(s) to parse each date into a datetime.datetime
I try to test a view, I receive a json request from the IPad, the format is:
req = {"custom_decks": [
{
"deck_name": "deck_test",
"updates_last_applied": "1406217357",
"created_date": 1406217380,
"slide_section_ids": [
1
],
"deck_id": 1
}
],
"custom_decks_to_delete": []
}
I checked this in jsonlint and it passed.
I post the req via:
response = self.client.post('/library/api/6.0/user/'+ uuid +
'/store_custom_dec/',content_type='application/json', data=req)
The view return "creation_success": false
The problem is the post method in view doesn't find the key custom_decks.
QueryDict: {u'{"custom_decks": [{"deck_id": 1, "slide_section_ids": [1],
"created_date":1406217380, "deck_name": "deck_test"}],
"custom_decks_to_delete": []}': [u'']}>
The problem is the post method in view doesn't find the key custom_decks.
Because it is converting my dict to QueryDict with one key.
I appreciate all helps.
Thanks
You're posting JSON, which is not the same as form-encoded data. You need to get the value of request.body and deserialize it:
data = json.loads(request.body)
custom_decks = data['custom_decks']
As I was having problems with getting JSON data from HttpRequest directly with the code of the other answer:
data = json.loads(request.body)
custom_decks = data['custom_decks']
error:
the JSON object must be str, not 'bytes'
Here is an update of the other answer for Python version >3:
json_str=((request.body).decode('utf-8'))
json_obj=json.loads(json_str)
Regarding decode('utf-8'), as mention in:
RFC 4627:
"JSON text shall be encoded in Unicode. The default encoding is
UTF-8."
I attached the Python link referred to this specific problem for version >3.
http://bugs.python.org/issue10976
python 3.6 and django 2.0 :
post_json = json.loads(request.body)
custom_decks = post_json.get("custom_decks")
json.loads(s, *, encoding=None,...)
Changed in version 3.6: s can now be of type bytes or bytearray. The input encoding should be UTF-8, UTF-16 or UTF-32.
From python 3.6 NO need request.body.decode('utf-8') .
Since HttpRequest has a read() method loading JSON from request is actually as simple as:
def post(self, request, *args, **kwargs):
import json
data = json.load(request)
return JsonResponse(data=data)
If you put this up as a view, you can test it and it'll echo any JSON you send back to you.
I'm trying to write a simple server frontend to a python3 application, using a restful JSON-based protocol. So far, bottle seems the best suited framework for the task (it supports python3, handles method dispatching in a nice way, and easily returns JSON.) The problem is parsing the JSON in the input request.
The documentation only mention request.fields and request.files, both I assume refer to multipart/form-data data. No mention of accessing the request data directly.
Peeking at the source code, I can see a request.body object of type BytesIO. json.load refuses to act on it directly, dying in the json lib with can't use a string pattern on a bytes-like object. The proper way to do it may be to first decode the bytes to unicode characters, according to whichever charset was specified in the Content-Type HTTP header. I don't know how to do that; I can see a StringIO class and assume it may hold a buffer of characters instead of bytes, but see no way of decoding a BytesIO to a StringIO, if this is even possible at all.
Of course, it may also be possible to read the BytesIO object into a bytestring, then decode it into a string before passing it to the JSON decoder, but if I understand correctly, that breaks the nice buffering behavior of the whole thing.
Or is there any better way to do it ?
It seems that io.TextIOWrapper from the standard library does the trick !
def parse(request):
encoding = ... #get encoding from headers
return json.load(TextIOWrapper(request.body, encoding=encoding))
Here's what I do to read in json on a RESTful service with Python3 and Bottle:
import bson.json_util as bson_json
#app.post('/location/API')
def post_json_example():
"""
param: _id, value
return: I usually return something like {"status": "successful", "message": "discription"}
"""
query_string = bottle.request.query.json
query_dict = bson_json.loads(query_string)
_id = query_dict['_id']
value = query_dict['value']
Then to Test
from python3 interpreter, import requests
s = request.Session()
r = s.post('http://youserver.com:8080/location/API?json
{"_id":"540a16663dafb492a0a7626c","value":"test"}')
use r.text to verify what was returned.
I wrote an helper to use the good idea of b0fh.
After 2 weeks on response.json analyzing, I connect to StackOver Flow and understand that we need a work around
Here is:
def json_app_rqt():
# about request
request.accept = 'application/json, text/plain; charset=utf-8'
def json_app_resp():
# about response
response.headers['Access-Control-Allow-Origin'] = _allow_origin
response.headers['Access-Control-Allow-Methods'] = _allow_methods
# response.headers['Access-Control-Allow-Headers'] = _allow_headers
response.headers['Content-Type'] = 'application/json; charset=utf-8'
def json_app():
json_app_rqt()
json_app_resp()
def get_json_request(rqt):
with TextIOWrapper(rqt.body, encoding = "UTF-8") as json_wrap:
json_text = ''.join(json_wrap.readlines())
json_data = json.loads(json_text)
return json_data
For the using, we cand do:
if __name__ == "__main__":
json_app()
#post("/train_control/:control")
def do_train_control(control):
json_app_resp()
data = get_json_request(request)
print(json.dumps(data))
return data
Thanks to all