I'm getting the following error when trying to parse a json response
expected string or buffer
Within my Django model I have the following:
def get_batch_prediction(self):
client = boto3.client('machinelearning', region_name=settings.region, aws_access_key_id=settings.aws_access_key_id, aws_secret_access_key=settings.aws_secret_access_key)
return client.get_batch_prediction(
BatchPredictionId=str(self.id)
)
I then call it like so
batch = BatchPrediction.objects.get(id=batch_id)
response = batch.get_batch_prediction()
response = json.loads(response)
I know the response is json so I expected this to change it to a dictionary but, instead, I get the error above.
What's going on?
The boto3 docs suggest that get_batch_prediction returns a dictionary not a string. You shouldn't have to use json.loads().
Related
I am making a curl call to rest api visa curl in groovy. Response is coming fine but the response is very large, it is a 17MB of data, following is my script :
def converter = "curl.......'"
def initialSize = 4096
def out = new ByteArrayOutputStream(initialSize)
def err = new ByteArrayOutputStream(initialSize)
def process = [ 'bash', '-c', converter].execute()
process.consumeProcessOutput(out, err)
process.waitFor()
Curl response is coming fine, when I print response on console ,store in variable out, it gives response data where it is not neat json as I see some "/n" characters. When I write this to file then I dont see any new line and neat json, all I see data in one line in key value format.
{"key1":"value1","key2":"value2",} in one huge line only
This is when i view in sublime. Now I want to convert this to pretty json and write neatly into file.I tried following to approaches but both prints empty ({ }) in console and in file.
def json = JsonOutput.toJson(out)
println new JsonBuilder(out).toPrettyString()
What did I miss?
I am trying to use groovy libraries only.
UPDATE:
As i try to debug, i found that it may be because all JSON parsers expect string but my output is ByteArrayOutputStream. But now how can I convert the out to string ? I tried out.toString and out.text, it does not work.
Use StringWriter instead of ByteArrayOutputStream
Then JsonOutput.prettyPrint( stringWriter.toString() )
My question
I tried many times to get information from url, In the website information is written in json form. And I found that 1,2,3 is wrong form.
what does request.get return?? and i wonder why 1, 2 and 3 does not work.
And how can I turn json in to dictionary form( not like fourth case)? I want to know other ways too. different from
response = response.json()
My codes(1,2,3 Wrong and 4 is correct)
1.TypeError: the JSON object must be str, bytes or bytearray, not Response
response = requests.get(URL)
response = json.loads(response)
2.TypeError: the JSON object must be str, bytes or bytearray, not Response
response = requests.get(URL)
response = json.load(response)
3.AttributeError: 'Response' object has no attribute 'read'
response = requests.get(URL)
response = response.read()
correct one
response = requests.get(URL)
response = response.read()
what does request.get return?
According to the documentation, it returns a requests.Response object.
And how can I turn json in to dictionary form
What's wrong with response = response.json()?
Your attempts to use json.load() and json.loads() failed because you passed it a Response object, instead of a str, bytes or bytearray. Be sure to fully read the documentation.
response.text gives you a str of the response, which is what you are really after. response.read() raises an AttributeError because there is no read() method for requests.Response objects. Again, make sure you fully read the documentation.
This is what you want: json.loads(response.text).
I am trying to read json response from this link. But its not working! I get the following error:
ValueError: No JSON object could be decoded.
Here is the code I've tried:
import urllib2, json
a = urllib2.urlopen('https://www.googleapis.com/pagespeedonline/v3beta1/mobileReady?key=AIzaSyDkEX-f1JNLQLC164SZaobALqFv4PHV-kA&screenshot=true&snapshots=true&locale=en_US&url=https://www.economicalinsurance.com/en/&strategy=mobile&filter_third_party_resources=false&callback=_callbacks_._DElanZU7Xh1K')
data = json.loads(a)
I made these changes:
import requests, json
r=requests.get('https://www.googleapis.com/pagespeedonline/v3beta1/mobileReady?key=AIzaSyDkEX-f1JNLQLC164SZaobALqFv4PHV-kA&screenshot=true&snapshots=true&locale=en_US&url=https://www.economicalinsurance.com/en/&strategy=mobile&filter_third_party_resources=false')
json_data = json.loads(r.text)
print json_data['ruleGroups']['USABILITY']['score']
A Quick question - Construct Image link .
I able to get here : -
from selenium import webdriver
txt = json_data['screenshot']['data']
txt = str(txt).replace('-','/').replace('_','/')
#then in order to construct the image link i tried : -
image_link = 'data:image/jpeg;base64,'+txt
driver = webdriver.Firefox()
driver.get(image_link)
The problem is i am not getting the image, also the len(object_original) as compared len(image_link) differs . Could anybody please advise the right elements missing in my constructed image link ?. Thank you
Here is API link - https://www.google.co.uk/webmasters/tools/mobile-friendly/ Sorry added it late .
Two corrections need to be made to your code:
The url was corrected (as mentioned by Felix Kling here). You have to remove the callback parameter from the GET request you were sending.
Also, if you check the type of the response that you were fetching earlier you'll notice that it wasn't a string. It was <type 'instance'>. And since json.loads() accepts a string as a parameter variable you would've got another error. Therefore, use a.read() to fetch the response data in string.
Hence, this should be your code:
import urllib2, json
a = urllib2.urlopen('https://www.googleapis.com/pagespeedonline/v3beta1/mobileReady?key=AIzaSyDkEX-f1JNLQLC164SZaobALqFv4PHV-kA&screenshot=true&snapshots=true&locale=en_US&url=https://www.economicalinsurance.com/en/&strategy=mobile&filter_third_party_resources=false')
data = json.loads(a.read())
Answer to your second query (regarding the image) is:
from base64 import decodestring
arr = json_data['screenshot']['data']
arr = arr.replace("_", "/")
arr = arr.replace("-","+")
fh = open("imageToSave.jpeg", "wb")
fh.write(str(arr).decode('base64'))
fh.close()
Here, is the image you were trying to fetch - Link
Felix Kling is right about the address, but I also created a variable that holds the URL. You can try this out to and it should work:
import urllib2, json
url = "https://www.googleapis.com/pagespeedonline/v3beta1/mobileReady?key=AIzaSyDkEX-f1JNLQLC164SZaobALqFv4PHV-kA&screenshot=true&snapshots=true&locale=en_US&url=https://www.economicalinsurance.com/en/&strategy=mobile&filter_third_party_resources=false"
response = urllib2.urlopen(url)
data = json.loads(response.read())
print data
I'm trying to use context.expand to get response and specific content from within the response,
def response = context.expand( '${Ranks of documents with SSE hits reflects scores#Response}' )
I also need to get specific detail from within the response, say if response has an array of ranks:
"ranks":[2234, 1234]
How can get both values of ranks?
You can use JsonSlurper from the groovy script testStep, supposing you get the follow JSON:
{"ranks":[2234, 1234]}
from your code:
def response = context.expand( '${Ranks of documents with SSE hits reflects scores#Response}')
You can use the JsonSlurper as follows to get your "ranks" values:
import groovy.json.JsonSlurper
// here I use this JSON as example, but you can
// parse directly your response which you get with context.expand
def response = '{"ranks":[2234, 1234]}'
def jsonRoot = new JsonSlurper().parseText(response)
log.info jsonRoot.ranks[0]
log.info jsonRoot.ranks[1]
Hope this helps,
Internally, SoapUI will convert almost anything to XML. You could grab just that node with ${step_name#ResponseAsXml}, and then parse it however you need to. Something like:
def ranksString = context.expand( '${Ranks of documents with SSE hits reflects scores#ResponseAsXml#//*:ranks}' )
def ranksArray = ranksString.split(",").trim()
log.info ranksArray[0]
log.info ranksArray[1]
I try to test a view, I receive a json request from the IPad, the format is:
req = {"custom_decks": [
{
"deck_name": "deck_test",
"updates_last_applied": "1406217357",
"created_date": 1406217380,
"slide_section_ids": [
1
],
"deck_id": 1
}
],
"custom_decks_to_delete": []
}
I checked this in jsonlint and it passed.
I post the req via:
response = self.client.post('/library/api/6.0/user/'+ uuid +
'/store_custom_dec/',content_type='application/json', data=req)
The view return "creation_success": false
The problem is the post method in view doesn't find the key custom_decks.
QueryDict: {u'{"custom_decks": [{"deck_id": 1, "slide_section_ids": [1],
"created_date":1406217380, "deck_name": "deck_test"}],
"custom_decks_to_delete": []}': [u'']}>
The problem is the post method in view doesn't find the key custom_decks.
Because it is converting my dict to QueryDict with one key.
I appreciate all helps.
Thanks
You're posting JSON, which is not the same as form-encoded data. You need to get the value of request.body and deserialize it:
data = json.loads(request.body)
custom_decks = data['custom_decks']
As I was having problems with getting JSON data from HttpRequest directly with the code of the other answer:
data = json.loads(request.body)
custom_decks = data['custom_decks']
error:
the JSON object must be str, not 'bytes'
Here is an update of the other answer for Python version >3:
json_str=((request.body).decode('utf-8'))
json_obj=json.loads(json_str)
Regarding decode('utf-8'), as mention in:
RFC 4627:
"JSON text shall be encoded in Unicode. The default encoding is
UTF-8."
I attached the Python link referred to this specific problem for version >3.
http://bugs.python.org/issue10976
python 3.6 and django 2.0 :
post_json = json.loads(request.body)
custom_decks = post_json.get("custom_decks")
json.loads(s, *, encoding=None,...)
Changed in version 3.6: s can now be of type bytes or bytearray. The input encoding should be UTF-8, UTF-16 or UTF-32.
From python 3.6 NO need request.body.decode('utf-8') .
Since HttpRequest has a read() method loading JSON from request is actually as simple as:
def post(self, request, *args, **kwargs):
import json
data = json.load(request)
return JsonResponse(data=data)
If you put this up as a view, you can test it and it'll echo any JSON you send back to you.