I've been trying to update a small Python library called libpynexmo to work with Python 3.
I've been stuck on this function:
def send_request_json(self, request):
url = request
req = urllib.request.Request(url=url)
req.add_header('Accept', 'application/json')
try:
return json.load(urllib.request.urlopen(req))
except ValueError:
return False
When it gets to this, json responds with:
TypeError: the JSON object must be str, not 'bytes'
I read in a few places that for json.load you should pass objects (In this case an HTTPResponse object) with a .read() attached, but it doesn't work on HTTPResponse objects.
I'm at a loss as to where to go with this next, but being that my entire 1500 line script is freshly converted to Python 3, I don't feel like going back to 2.7.
Facing the same problem I solve it using decode()
...
rawreply = connection.getresponse().read()
reply = json.loads(rawreply.decode())
I recently wrote a small function to send Nexmo messages. Unless you need the full functionality of the libpynexmo code, this should do the job for you. And if you want to continue overhauling libpynexmo, just copy this code. The key is utf8 encoding.
If you want to send any other fields with your message, the full documentation for what you can include with a nexmo outbound message is here
Python 3.4 tested Nexmo outbound (JSON):
def nexmo_sendsms(api_key, api_secret, sender, receiver, body):
"""
Sends a message using Nexmo.
:param api_key: Nexmo provided api key
:param api_secret: Nexmo provided secrety key
:param sender: The number used to send the message
:param receiver: The number the message is addressed to
:param body: The message body
:return: Returns the msgid received back from Nexmo after message has been sent.
"""
msg = {
'api_key': api_key,
'api_secret': api_secret,
'from': sender,
'to': receiver,
'text': body
}
nexmo_url = 'https://rest.nexmo.com/sms/json'
data = urllib.parse.urlencode(msg)
binary_data = data.encode('utf8')
req = urllib.request.Request(nexmo_url, binary_data)
response = urllib.request.urlopen(req)
result = json.loads(response.readall().decode('utf-8'))
return result['messages'][0]['message-id']
I met the problem as well and now it pass
import json
import urllib.request as ur
import urllib.parse as par
html = ur.urlopen(url).read()
print(type(html))
data = json.loads(html.decode('utf-8'))
Since you are getting a HTTPResponse, you can use Tornado.escape and its json_decode() to convert the JSON string into a dictionary:
from tornado import escape
body = escape.json_decode(body)
From the manual:
tornado.escape.json_decode(value)
Returns Python objects for the given JSON string.
Related
I'm setting up a request assertion with request module and python assert keyword but getting
AttributeError: 'dict' object has no attribute 'documentation_url'
when I try to assert a string in json response. How do I assert something within the json response and when the condition is true, it should print out something?
import requests
import pprint
URL = 'https://github.com/timeline.json'
def get_github_json(URL):
response = requests.get(URL).json()
return response
assert get_github_json(URL).documentation_url == 'https://developer.github.com/v3/activity/events/#list-public-events'
The json response looks like this:
{'documentation_url': 'https://developer.github.com/v3/activity/events/#list-public-events',
'message': 'Hello there, wayfaring stranger. If you’re reading this then you '
'probably didn’t see our blog post a couple of years back '
'announcing that this API would go away: http://git.io/17AROg Fear '
'not, you should be able to get what you need from the shiny new '
'Events API instead.'
}
Hi because it's a dictionary then you would have to get the value with a key.
We know response is a dictionary, so in this case when you want the value from the documentation_url, we would need to do it like this:
def get_github_json(url):
response = requests.get(url).json()
return response
assert get_github_json(url)['documentation_url'] # <---- your are getting the value by giving the key
If you try and print out response['documentation_url'] then you would get this result:
https://developer.github.com/v3/activity/events/#list-public-events
I am trying to familiarize myself with Python, coming from a heavy PHP background. I want to create a JSON object that returns the following:
Endpoint: /dashboard
Request params:{
method:”get”,
headers:headers,
url:string,
data:{
userId: string,
date: int
}
}
Response: (JSON Object)
{
code: 200/400,
message: String,
data: {
totalCal:
remainingCal:
}
}
Essentially connecting to the database and returning a 200 if the data matches, a 400 if i hit any errors. I came up with this so far, but I'd appreciate any input on how I could perform this better!
#app.route('/dashboard', methods=['GET'])
def availCalories():
cur = conn.cursor()
cur.execute('''SELECT remaining_cal FROM daily_cal WHERE user_id = 1 AND day= curdate();''')# WHERE userID=userID in session???
row_headers=[x[0] for x in cur.description] #EXTRACT ROW HEADERS
rv = cur.fetchall()
json_data=[]
for result in rv:
json_data.append(dict(zip(row_headers,result)))
return json.dumps(json_data)
I've been teaching myself Python for only 3 months now, so a lot of this is trial and error, but I find myself hitting a wall with this!
Thanks for your time in advance!
You are using flask so you can also investigate the jsonify method, which is convenient and common:
flask.json.jsonify(*args, **kwargs)
This function wraps dumps() to add a few enhancements that make life easier. It turns the JSON output into a Response object with the application/json mimetype. For convenience, it also converts multiple arguments into an array or multiple keyword arguments into a dict. This means that both jsonify(1,2,3) and jsonify([1,2,3]) serialize to [1,2,3].
I'm trying to use context.expand to get response and specific content from within the response,
def response = context.expand( '${Ranks of documents with SSE hits reflects scores#Response}' )
I also need to get specific detail from within the response, say if response has an array of ranks:
"ranks":[2234, 1234]
How can get both values of ranks?
You can use JsonSlurper from the groovy script testStep, supposing you get the follow JSON:
{"ranks":[2234, 1234]}
from your code:
def response = context.expand( '${Ranks of documents with SSE hits reflects scores#Response}')
You can use the JsonSlurper as follows to get your "ranks" values:
import groovy.json.JsonSlurper
// here I use this JSON as example, but you can
// parse directly your response which you get with context.expand
def response = '{"ranks":[2234, 1234]}'
def jsonRoot = new JsonSlurper().parseText(response)
log.info jsonRoot.ranks[0]
log.info jsonRoot.ranks[1]
Hope this helps,
Internally, SoapUI will convert almost anything to XML. You could grab just that node with ${step_name#ResponseAsXml}, and then parse it however you need to. Something like:
def ranksString = context.expand( '${Ranks of documents with SSE hits reflects scores#ResponseAsXml#//*:ranks}' )
def ranksArray = ranksString.split(",").trim()
log.info ranksArray[0]
log.info ranksArray[1]
I try to test a view, I receive a json request from the IPad, the format is:
req = {"custom_decks": [
{
"deck_name": "deck_test",
"updates_last_applied": "1406217357",
"created_date": 1406217380,
"slide_section_ids": [
1
],
"deck_id": 1
}
],
"custom_decks_to_delete": []
}
I checked this in jsonlint and it passed.
I post the req via:
response = self.client.post('/library/api/6.0/user/'+ uuid +
'/store_custom_dec/',content_type='application/json', data=req)
The view return "creation_success": false
The problem is the post method in view doesn't find the key custom_decks.
QueryDict: {u'{"custom_decks": [{"deck_id": 1, "slide_section_ids": [1],
"created_date":1406217380, "deck_name": "deck_test"}],
"custom_decks_to_delete": []}': [u'']}>
The problem is the post method in view doesn't find the key custom_decks.
Because it is converting my dict to QueryDict with one key.
I appreciate all helps.
Thanks
You're posting JSON, which is not the same as form-encoded data. You need to get the value of request.body and deserialize it:
data = json.loads(request.body)
custom_decks = data['custom_decks']
As I was having problems with getting JSON data from HttpRequest directly with the code of the other answer:
data = json.loads(request.body)
custom_decks = data['custom_decks']
error:
the JSON object must be str, not 'bytes'
Here is an update of the other answer for Python version >3:
json_str=((request.body).decode('utf-8'))
json_obj=json.loads(json_str)
Regarding decode('utf-8'), as mention in:
RFC 4627:
"JSON text shall be encoded in Unicode. The default encoding is
UTF-8."
I attached the Python link referred to this specific problem for version >3.
http://bugs.python.org/issue10976
python 3.6 and django 2.0 :
post_json = json.loads(request.body)
custom_decks = post_json.get("custom_decks")
json.loads(s, *, encoding=None,...)
Changed in version 3.6: s can now be of type bytes or bytearray. The input encoding should be UTF-8, UTF-16 or UTF-32.
From python 3.6 NO need request.body.decode('utf-8') .
Since HttpRequest has a read() method loading JSON from request is actually as simple as:
def post(self, request, *args, **kwargs):
import json
data = json.load(request)
return JsonResponse(data=data)
If you put this up as a view, you can test it and it'll echo any JSON you send back to you.
I'm trying to write a simple server frontend to a python3 application, using a restful JSON-based protocol. So far, bottle seems the best suited framework for the task (it supports python3, handles method dispatching in a nice way, and easily returns JSON.) The problem is parsing the JSON in the input request.
The documentation only mention request.fields and request.files, both I assume refer to multipart/form-data data. No mention of accessing the request data directly.
Peeking at the source code, I can see a request.body object of type BytesIO. json.load refuses to act on it directly, dying in the json lib with can't use a string pattern on a bytes-like object. The proper way to do it may be to first decode the bytes to unicode characters, according to whichever charset was specified in the Content-Type HTTP header. I don't know how to do that; I can see a StringIO class and assume it may hold a buffer of characters instead of bytes, but see no way of decoding a BytesIO to a StringIO, if this is even possible at all.
Of course, it may also be possible to read the BytesIO object into a bytestring, then decode it into a string before passing it to the JSON decoder, but if I understand correctly, that breaks the nice buffering behavior of the whole thing.
Or is there any better way to do it ?
It seems that io.TextIOWrapper from the standard library does the trick !
def parse(request):
encoding = ... #get encoding from headers
return json.load(TextIOWrapper(request.body, encoding=encoding))
Here's what I do to read in json on a RESTful service with Python3 and Bottle:
import bson.json_util as bson_json
#app.post('/location/API')
def post_json_example():
"""
param: _id, value
return: I usually return something like {"status": "successful", "message": "discription"}
"""
query_string = bottle.request.query.json
query_dict = bson_json.loads(query_string)
_id = query_dict['_id']
value = query_dict['value']
Then to Test
from python3 interpreter, import requests
s = request.Session()
r = s.post('http://youserver.com:8080/location/API?json
{"_id":"540a16663dafb492a0a7626c","value":"test"}')
use r.text to verify what was returned.
I wrote an helper to use the good idea of b0fh.
After 2 weeks on response.json analyzing, I connect to StackOver Flow and understand that we need a work around
Here is:
def json_app_rqt():
# about request
request.accept = 'application/json, text/plain; charset=utf-8'
def json_app_resp():
# about response
response.headers['Access-Control-Allow-Origin'] = _allow_origin
response.headers['Access-Control-Allow-Methods'] = _allow_methods
# response.headers['Access-Control-Allow-Headers'] = _allow_headers
response.headers['Content-Type'] = 'application/json; charset=utf-8'
def json_app():
json_app_rqt()
json_app_resp()
def get_json_request(rqt):
with TextIOWrapper(rqt.body, encoding = "UTF-8") as json_wrap:
json_text = ''.join(json_wrap.readlines())
json_data = json.loads(json_text)
return json_data
For the using, we cand do:
if __name__ == "__main__":
json_app()
#post("/train_control/:control")
def do_train_control(control):
json_app_resp()
data = get_json_request(request)
print(json.dumps(data))
return data
Thanks to all