Json parse error using POST in django rest api - json

I am trying to implement a simple GET/POST api via Django REST framework
views.py
class cuser(APIView):
def post(self, request):
stream = BytesIO(request.DATA)
json = JSONParser().parse(stream)
return Response()
urls.py
from django.conf.urls import patterns, url
from app import views
urlpatterns = patterns('',
url(r'^challenges/',views.getall.as_view() ),
url(r'^cuser/' , views.cuser.as_view() ),
)
I am trying to POST some json to /api/cuser/ (api is namespace in my project's urls.py ) ,
the JSON
{
"username" : "abhishek",
"email" : "john#doe.com",
"password" : "secretpass"
}
I tried from both Browseable API page and httpie ( A python made tool similar to curl)
httpie command
http --json POST http://localhost:58601/api/cuser/ username=abhishek email=john#doe.com password=secretpass
but I am getting JSON parse error :
JSON parse error - Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
Whole Debug message using --verbose --debug
POST /api/cuser/ HTTP/1.1
Content-Length: 75
Accept-Encoding: gzip, deflate
Host: localhost:55392
Accept: application/json
User-Agent: HTTPie/0.8.0
Connection: keep-alive
Content-Type: application/json; charset=utf-8
{"username": "abhishek", "email": "john#doe.com", "password": "aaezaakmi1"}
HTTP/1.0 400 BAD REQUEST
Date: Sat, 24 Jan 2015 09:40:03 GMT
Server: WSGIServer/0.1 Python/2.7.9
Vary: Accept, Cookie
Content-Type: application/json
Allow: POST, OPTIONS
{"detail":"JSON parse error - Expecting property name enclosed in double quotes: line 1 column 2 (char 1)"}

The problem that you are running into is that your request is already being parsed, and you are trying to parse it a second time.
From "How the parser is determined"
The set of valid parsers for a view is always defined as a list of classes. When request.data is accessed, REST framework will examine the Content-Type header on the incoming request, and determine which parser to use to parse the request content.
In your code you are accessing request.DATA, which is the 2.4.x equaivalent of request.data. So your request is being parsed as soon as you call that, and request.DATA is actually returning the dictionary that you were expecting to parse.
json = request.DATA
is really all you need to parse the incoming JSON data. You were really passing a Python dictionary into json.loads, which does not appear to be able to parse it, and that is why you were getting your error.

I arrived at this post via Google for
"detail": "JSON parse error - Expecting property name enclosed in double-quotes":
Turns out you CANNOT have a trailing comma in JSON.
So if you are getting this error you may need to change a post like this:
{
"username" : "abhishek",
"email" : "john#doe.com",
"password" : "secretpass",
}
to this:
{
"username" : "abhishek",
"email" : "john#doe.com",
"password" : "secretpass"
}
Note the removed comma after the last property in the JSON object.

Basically, whenever you are trying to make a post request with requests lib, This library also contains json argument which is ignored in the case when data argument is set to files or data. So basically when json argument is set with json data. Headers are set asContent-Type: application/json. Json argument basically encodes data sends into a json format. So that at DRF particularly is able to parse json data. Else in case of only data argument it is been treated as form-encoded
requests.post(url, json={"key1":"value1"})
you can find more here request.post complicated post methods

Related

line separator(LR) escape issue in json request body with WebClient

I have a project about graphql (language is JAVA), in some json value I need to add the LF(\n) in the content.
If I add the "\n" in java code, the body written is \n. No escape here.
WRITE: 273B POST /v1/graphql HTTP/1.1
user-agent: ReactorNetty/1.0.22
host: api.xxxx.com
Authorization: Bearer xxxxxxxx
Accept: application/json
Content-type: application/json
content-length: 894
WRITE: 894B {"query":"mutation {updateProduct(input: {..., description:\"ABC\nDEF\nGHI\", ... }){ product { id } } }","variables":{},"operationName":null}
ONCE I add the escaped one "\\n" in code and the body written from webclient is \\\\n. The double \\ is escaped...
WRITE: 894B {"query":"mutation {updateProduct(input: {..., description:\"ABC\\\\nDEF\\\\nGHI\", ... }){ product { id } } }","variables":{},"operationName":null}
WHY??? I hope the written one is \\n as properly escaped.
It's a bug? Or I missed some setting about webclient?
BTW, the client code is below (I'm using DGS framework also, but I think the issue is no related DGS)
HttpClient httpClient = HttpClient.create().wiretap(this.getClass().getCanonicalName(),LogLevel.DEBUG,AdvancedByteBufFormat.TEXTUAL); this.client = new WebClientGraphQLClient(WebClient.builder().baseUrl(endpoint).clientConnector(new ReactorClientHttpConnector(httpClient)).defaultHeader("Authorization", StringUtils.join("Bearer", " ", key)).build());

Robot Framework bypass HTTPError: 400 Client Error: Bad Request

I have a request which tend to upload a file, if a file with the same name already exists it throws a message that the file already exists. This can be considered as expected result and even though the error I would the test to pass as it is.
This is the code I am using:
Create Session mysession ${test_env}
&{headers} Create Dictionary Content-Type=application/json; charset=utf-8 Authorization=${token}
${json}= Catenate { "FileName": "File.txt", "Content": "PD94bWwg..", "UserId": "email.com" }
${value} Set Variable 2
${value} Convert To Integer ${value}
${json}= Evaluate json.loads('''${json}''') json
#Set To Dictionary ${json["FileName"]}
${json}= Evaluate json.dumps(${json}) json
${resp} POST url=${test_env}/api/nt data=${json} headers=${headers}
${log}= Log To Console ${resp.status_code} 400
Log To Console ${resp.content}
Status Should Be expected_status=any
The test stops at the POST request and does not want to read the expected_status=any and consider the test as pass.
I would appreciate any hints on how to make it pass.
Below code will verify the 400 error and will continue further execution
Run Keyword And Expect Error HTTPError: 400* POST url=${test_env}/api/nt data=${json} headers=${headers}

Karate properties.json throws ReferenceError when Reading Properties in *.feature file

Having updated Karate from 0.6.2 to 0.9.5 recently I've had a number of ReferenceError's w.r.t the properties.json I've used throughout my test cases.
I've the following setup:
test-properties.json
{
"headers": {
"x-client-ip": "192.168.3.1",
"x-forwarded-for": "192.168.3.1"
}
}
test-auth.feature
Background:
* def props = read('properties/test-properties.json')
I then use props further down in my first scenario:
And header User-Agent = props.headers.Accept-Language
And header X-Forwarded-For = props.headers.x-forwarded-for
However, when running this I get the following issue:
com.intuit.karate.exception.KarateException: test-auth.feature:14 - javascript evaluation failed: props.headers.Accept-Language, ReferenceError: "Language" is not defined in <eval> at line number 1
I've tried adding the properties file into the same package as the test-auth.feature to no avail. The issue seems to be with reading the json file. I'm aware Karate 0.6.2 could evaluate the file type and parse it internally in its native format. Is this still the case? If not, what is the solution to reading from properties.json in Karate 0.9.5.
Nothing should have changed when it comes to reading JSON files. Karate evaluates the RHS as JS, so I think this is the solution:
And header User-Agent = props.headers['Accept-Language']
And header X-Forwarded-For = props.headers['x-forwarded-for']
EDIT: this works for me:
* def props = { headers: { 'Accept-Language': 'foo', 'x-forwarded-for': 'bar' } }
* url 'http://httpbin.org/headers'
* header User-Agent = props.headers['Accept-Language']
* header X-Forwarded-For = props.headers['x-forwarded-for']
* method get
Resulting in:
1 > GET http://httpbin.org/headers
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Host: httpbin.org
1 > User-Agent: foo
1 > X-Forwarded-For: bar
So if you are still stuck, please follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

JSON string fails to convert to correct Lua table

I'm performing a request to a foreign API using luasec, lua-socket and converting the data, a JSON string, to a lua table with cjson. I've read the docs of said modules and unfortunately none of it helped me with my problem. Can't link more than 2 websites with current account, sorry.
Summary: I get the response and the appropiate string using the posted request function, when turning said string into a lua table via cjson.decode the output table isn't the desired one, it's a copy of my response header, which is not intentional.
The following code is how I do my request:
local function request (req_t)
local res_t = {}
resp = https.request {
url = const.API_URL .. req_t.url,
method = req_t.method,
headers = req_t.headers,
sink = ltn12.sink.table(res_t)
}
return table.concat(res_t), resp.headers, resp.code
end
Using the following call
local res, headers = request({ ... })
I receive the proper response as a string but my goal is to do data manipulation with it, so turning said response(string) to a lua table with
local resJson = cjson.decode(res)
Does not produce the correct output. It does produce a table which is exactly the same as my response header. Here is the following output from my terminal alongside the code
When out of function type is: string
Desired response in string:
{"total_photos":221926,"photo_downloads":"186029632.0"}
When out of function type is: string
Desired response in string:
{"total_photos":221926,"photo_downloads":"186029632.0"}
After decode, type is: table
server Cowboy
strict-transport-security max-age=31536000
access-control-allow-headers *
x-ratelimit-limit 50
x-ratelimit-remaining 46
x-cache-hits 0, 0
accept-ranges bytes
access-control-request-method *
x-request-id ee5a74fd-2b10-4f46-9c25-5cfc53aeac6c
access-control-expose-headers Link,X-Total,X-Per-Page,X-RateLimit-Limit,X-RateLimit-Remaining
content-type application/json
connection close
content-length 55
fastly-debug-digest f62d52c08b1ef74db89a66a0069f0a35c49e52230567905240dacf08c9ea1813
vary Origin
cache-control no-cache, no-store, must-revalidate
x-timer S1496524765.369880,VS0,VE111
x-cache MISS, MISS
x-served-by cache-iad2123-IAD, cache-mad9429-MAD
via 1.1 vegur, 1.1 varnish, 1.1 varnish
date Sat, 03 Jun 2017 21:19:25 GMT
age 0
access-control-allow-origin *
x-runtime 0.011667
Printing header
server Cowboy
strict-transport-security max-age=31536000
access-control-allow-headers *
x-ratelimit-limit 50
x-ratelimit-remaining 46
x-cache-hits 0, 0
accept-ranges bytes
access-control-request-method *
x-request-id ee5a74fd-2b10-4f46-9c25-5cfc53aeac6c
access-control-expose-headers Link,X-Total,X-Per-Page,X-RateLimit-Limit,X-RateLimit-Remaining
content-type application/json
connection close
content-length 55
fastly-debug-digest f62d52c08b1ef74db89a66a0069f0a35c49e52230567905240dacf08c9ea1813
vary Origin
cache-control no-cache, no-store, must-revalidate
x-timer S1496524765.369880,VS0,VE111
x-cache MISS, MISS
x-served-by cache-iad2123-IAD, cache-mad9429-MAD
via 1.1 vegur, 1.1 varnish, 1.1 varnish
date Sat, 03 Jun 2017 21:19:25 GMT
age 0
access-control-allow-origin *
x-runtime 0.011667
Function that produces said log
local res, headers = request({ ... })
print('When out of function type is: ' ..type(res) .. '\n')
print('Desired response in string:')
print(res .. '\n')
resJson = cjson.decode(res)
print('\nAfter decode, type is: ' .. type(resJson) .. '\n')
pTable(resJson)
print('\nPrinting header\n')
pTable(headers)
pTable is just a function to output a table to stdout.
Thanks in advance
Posted function and routines are correct. The problem was located in my print table function, which I somehow had hardcoded my headers.

JSONDecodeError: Expecting value: line 1 column 1

I am receiving this error in Python 3.5.1.
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Here is my code:
import json
import urllib.request
connection = urllib.request.urlopen('http://python-data.dr-chuck.net/comments_220996.json')
js = connection.read()
print(js)
info = json.loads(str(js))
If you look at the output you receive from print() and also in your Traceback, you'll see the value you get back is not a string, it's a bytes object (prefixed by b):
b'{\n "note":"This file .....
If you fetch the URL using a tool such as curl -v, you will see that the content type is
Content-Type: application/json; charset=utf-8
So it's JSON, encoded as UTF-8, and Python is considering it a byte stream, not a simple string. In order to parse this, you need to convert it into a string first.
Change the last line of code to this:
info = json.loads(js.decode("utf-8"))
in my case, some characters like " , :"'{}[] " maybe corrupt the JSON format, so use try json.loads(str) except to check your input