I have a project about graphql (language is JAVA), in some json value I need to add the LF(\n) in the content.
If I add the "\n" in java code, the body written is \n. No escape here.
WRITE: 273B POST /v1/graphql HTTP/1.1
user-agent: ReactorNetty/1.0.22
host: api.xxxx.com
Authorization: Bearer xxxxxxxx
Accept: application/json
Content-type: application/json
content-length: 894
WRITE: 894B {"query":"mutation {updateProduct(input: {..., description:\"ABC\nDEF\nGHI\", ... }){ product { id } } }","variables":{},"operationName":null}
ONCE I add the escaped one "\\n" in code and the body written from webclient is \\\\n. The double \\ is escaped...
WRITE: 894B {"query":"mutation {updateProduct(input: {..., description:\"ABC\\\\nDEF\\\\nGHI\", ... }){ product { id } } }","variables":{},"operationName":null}
WHY??? I hope the written one is \\n as properly escaped.
It's a bug? Or I missed some setting about webclient?
BTW, the client code is below (I'm using DGS framework also, but I think the issue is no related DGS)
HttpClient httpClient = HttpClient.create().wiretap(this.getClass().getCanonicalName(),LogLevel.DEBUG,AdvancedByteBufFormat.TEXTUAL); this.client = new WebClientGraphQLClient(WebClient.builder().baseUrl(endpoint).clientConnector(new ReactorClientHttpConnector(httpClient)).defaultHeader("Authorization", StringUtils.join("Bearer", " ", key)).build());
Related
I'm creating an email client using Golang, to send email with a CSV file attached. Everything is working fine except that in the received email attachment, I can see some unwanted extra characters at the end of the file.
My code snippet:
import (
"bytes"
"encoding/base64"
"fmt"
"mime/multipart"
"net/smtp"
...
)
func SendEmail(cfg Config) error {
body := bytes.NewBuffer(nil)
body.WriteString(fmt.Sprintf("From: %s\n", cfg.EmailFrom))
body.WriteString(fmt.Sprintf("To: %s\n", cfg.EmailTo))
body.WriteString(fmt.Sprintf("Subject: %s\n", cfg.EmailSubject))
// csv file to attach
fileContents := `column1,column2,column3\nAAA,BBB,CCC\nDDD,EEE,FFF\n`
fileContentBytes := []byte(fileContents)
body.WriteString("MIME-Version: 1.0\n")
writer := multipart.NewWriter(body)
boundary := writer.Boundary()
// attach file
body.WriteString("Content-Type: text/plain\n")
body.WriteString("Content-Transfer-Encoding: base64\n")
body.WriteString(fmt.Sprintf("Content-Disposition: attachment; filename=%s\n", "test-filename"))
encodedBytes := make([]byte, base64.StdEncoding.EncodedLen(len(fileContentBytes)))
base64.StdEncoding.Encode(encodedBytes, fileContentBytes)
body.Write(encodedBytes)
body.WriteString(fmt.Sprintf("\n--%s--", boundary))
err = smtp.SendMail(cfg.EmailSMTPHost+":"+cfg.EmailSMTPPort,
nil, cfg.EmailFrom, []string{cfg.EmailTo}, body.Bytes())
if err != nil {
return errors.Wrap(err, "smtp.SendMail failed")
}
return nil
}
Expected csv file:
column1,column2,column3
AAA,BBB,CCC
DDD,EEE,FFF
Obtained csv file:
column1,column2,column3
AAA,BBB,CCC
DDD,EEE,FFF
5k§xõí»ã}8
Anything wrong in the file contents encoding? Any help will be appreciated, thanks!
This code has at least two problems: missing empty line to separate MIME header and body and then adding some MIME boundary at the end even though this is no multipart mail. Currently the created mail looks like this:
From: me#example.com
To: you#example.com
Subject: test
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=test-filename
Y29sdW1uMSxjb2x1bW4yLGNvbHVtbjNcbkFBQSxCQkIsQ0NDXG5EREQsRUVFLEZGRlxu
--973d0754ef322150f1977af176c9e1917c6dea9dfa0390e8e99af038c086--
The wrong boundary at the end gets decoded as base64 with invalid base64 characters like "-" simply being ignored. This causes the garbage at the end of the output.
It should instead look like this as a single part. Note the missing (wrong) end-boundary and note the empty line between MIME header and body.
From: me#example.com
To: you#example.com
Subject: test
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=test-filename
Y29sdW1uMSxjb2x1bW4yLGNvbHVtbjNcbkFBQSxCQkIsQ0NDXG5EREQsRUVFLEZGRlxu
Alternatively it should be done as a multipart mail as shown below. Note the different Content-Type in the main MIME header.
From: me#example.com
To: you#example.com
Subject: test
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary=973d0754ef322150f1977af176c9e1917c6dea9dfa0390e8e99af038c086
--973d0754ef322150f1977af176c9e1917c6dea9dfa0390e8e99af038c086
Content-Type: text/plain
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=test-filename
Y29sdW1uMSxjb2x1bW4yLGNvbHVtbjNcbkFBQSxCQkIsQ0NDXG5EREQsRUVFLEZGRlxu
--973d0754ef322150f1977af176c9e1917c6dea9dfa0390e8e99af038c086--
I have a json log that expands to this:
JSON:
|-host : hostname
|-httpRequest
|-httpVersion : HTTP/1.1
|-headers
|-0
|-name: X-Forwarded-For
|-value: 1.1.1.1
|-1
|-name: X-Forwarded-Prot
|-value: https
|-2
|-name: X-Forwarded-Port
|-value: 443
|-httpMethod: post
|-action: allow
etc..
I would like to reformat it like this:
JSON:
|-host : hostname
|-httpRequest
|-httpVersion : HTTP/1.1
|-headers
|-X-Forwarded-For : 1.1.1.1
|-X-Forwarded-Prot : https
|-X-Forwarded-Port : 443
|-httpMethod: post
|-action: allow
Split will just take the last [#] name/value as it overwrites the previous.
I am pretty sure this will need a ruby code block, but I haven't had luck following along with ruby code I have found online for similar scenarios.
I think the main issue here/difference with other article/answers is that it's not just a pure flatten. But need to rearrange the name:value a bit as well.
This seems to do the trick as well:
Newfield
ruby {
code => '
event.get("[#metadata][json][httpRequest][headers]").each do |header|
event.set("[newfield][#{header["name"]}]", header["value"])
end
'
}
Same field
ruby {
code => '
headerHash = {}
event.get("[#metadata][json][httpRequest][headers]").each do |header|
headerHash[header["name"]] = header["value"]
end
event.set("[#metadata][json][httpRequest][headers]", headerHash)
'
}
Having updated Karate from 0.6.2 to 0.9.5 recently I've had a number of ReferenceError's w.r.t the properties.json I've used throughout my test cases.
I've the following setup:
test-properties.json
{
"headers": {
"x-client-ip": "192.168.3.1",
"x-forwarded-for": "192.168.3.1"
}
}
test-auth.feature
Background:
* def props = read('properties/test-properties.json')
I then use props further down in my first scenario:
And header User-Agent = props.headers.Accept-Language
And header X-Forwarded-For = props.headers.x-forwarded-for
However, when running this I get the following issue:
com.intuit.karate.exception.KarateException: test-auth.feature:14 - javascript evaluation failed: props.headers.Accept-Language, ReferenceError: "Language" is not defined in <eval> at line number 1
I've tried adding the properties file into the same package as the test-auth.feature to no avail. The issue seems to be with reading the json file. I'm aware Karate 0.6.2 could evaluate the file type and parse it internally in its native format. Is this still the case? If not, what is the solution to reading from properties.json in Karate 0.9.5.
Nothing should have changed when it comes to reading JSON files. Karate evaluates the RHS as JS, so I think this is the solution:
And header User-Agent = props.headers['Accept-Language']
And header X-Forwarded-For = props.headers['x-forwarded-for']
EDIT: this works for me:
* def props = { headers: { 'Accept-Language': 'foo', 'x-forwarded-for': 'bar' } }
* url 'http://httpbin.org/headers'
* header User-Agent = props.headers['Accept-Language']
* header X-Forwarded-For = props.headers['x-forwarded-for']
* method get
Resulting in:
1 > GET http://httpbin.org/headers
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Host: httpbin.org
1 > User-Agent: foo
1 > X-Forwarded-For: bar
So if you are still stuck, please follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
I'm performing a request to a foreign API using luasec, lua-socket and converting the data, a JSON string, to a lua table with cjson. I've read the docs of said modules and unfortunately none of it helped me with my problem. Can't link more than 2 websites with current account, sorry.
Summary: I get the response and the appropiate string using the posted request function, when turning said string into a lua table via cjson.decode the output table isn't the desired one, it's a copy of my response header, which is not intentional.
The following code is how I do my request:
local function request (req_t)
local res_t = {}
resp = https.request {
url = const.API_URL .. req_t.url,
method = req_t.method,
headers = req_t.headers,
sink = ltn12.sink.table(res_t)
}
return table.concat(res_t), resp.headers, resp.code
end
Using the following call
local res, headers = request({ ... })
I receive the proper response as a string but my goal is to do data manipulation with it, so turning said response(string) to a lua table with
local resJson = cjson.decode(res)
Does not produce the correct output. It does produce a table which is exactly the same as my response header. Here is the following output from my terminal alongside the code
When out of function type is: string
Desired response in string:
{"total_photos":221926,"photo_downloads":"186029632.0"}
When out of function type is: string
Desired response in string:
{"total_photos":221926,"photo_downloads":"186029632.0"}
After decode, type is: table
server Cowboy
strict-transport-security max-age=31536000
access-control-allow-headers *
x-ratelimit-limit 50
x-ratelimit-remaining 46
x-cache-hits 0, 0
accept-ranges bytes
access-control-request-method *
x-request-id ee5a74fd-2b10-4f46-9c25-5cfc53aeac6c
access-control-expose-headers Link,X-Total,X-Per-Page,X-RateLimit-Limit,X-RateLimit-Remaining
content-type application/json
connection close
content-length 55
fastly-debug-digest f62d52c08b1ef74db89a66a0069f0a35c49e52230567905240dacf08c9ea1813
vary Origin
cache-control no-cache, no-store, must-revalidate
x-timer S1496524765.369880,VS0,VE111
x-cache MISS, MISS
x-served-by cache-iad2123-IAD, cache-mad9429-MAD
via 1.1 vegur, 1.1 varnish, 1.1 varnish
date Sat, 03 Jun 2017 21:19:25 GMT
age 0
access-control-allow-origin *
x-runtime 0.011667
Printing header
server Cowboy
strict-transport-security max-age=31536000
access-control-allow-headers *
x-ratelimit-limit 50
x-ratelimit-remaining 46
x-cache-hits 0, 0
accept-ranges bytes
access-control-request-method *
x-request-id ee5a74fd-2b10-4f46-9c25-5cfc53aeac6c
access-control-expose-headers Link,X-Total,X-Per-Page,X-RateLimit-Limit,X-RateLimit-Remaining
content-type application/json
connection close
content-length 55
fastly-debug-digest f62d52c08b1ef74db89a66a0069f0a35c49e52230567905240dacf08c9ea1813
vary Origin
cache-control no-cache, no-store, must-revalidate
x-timer S1496524765.369880,VS0,VE111
x-cache MISS, MISS
x-served-by cache-iad2123-IAD, cache-mad9429-MAD
via 1.1 vegur, 1.1 varnish, 1.1 varnish
date Sat, 03 Jun 2017 21:19:25 GMT
age 0
access-control-allow-origin *
x-runtime 0.011667
Function that produces said log
local res, headers = request({ ... })
print('When out of function type is: ' ..type(res) .. '\n')
print('Desired response in string:')
print(res .. '\n')
resJson = cjson.decode(res)
print('\nAfter decode, type is: ' .. type(resJson) .. '\n')
pTable(resJson)
print('\nPrinting header\n')
pTable(headers)
pTable is just a function to output a table to stdout.
Thanks in advance
Posted function and routines are correct. The problem was located in my print table function, which I somehow had hardcoded my headers.
I am trying to implement a simple GET/POST api via Django REST framework
views.py
class cuser(APIView):
def post(self, request):
stream = BytesIO(request.DATA)
json = JSONParser().parse(stream)
return Response()
urls.py
from django.conf.urls import patterns, url
from app import views
urlpatterns = patterns('',
url(r'^challenges/',views.getall.as_view() ),
url(r'^cuser/' , views.cuser.as_view() ),
)
I am trying to POST some json to /api/cuser/ (api is namespace in my project's urls.py ) ,
the JSON
{
"username" : "abhishek",
"email" : "john#doe.com",
"password" : "secretpass"
}
I tried from both Browseable API page and httpie ( A python made tool similar to curl)
httpie command
http --json POST http://localhost:58601/api/cuser/ username=abhishek email=john#doe.com password=secretpass
but I am getting JSON parse error :
JSON parse error - Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
Whole Debug message using --verbose --debug
POST /api/cuser/ HTTP/1.1
Content-Length: 75
Accept-Encoding: gzip, deflate
Host: localhost:55392
Accept: application/json
User-Agent: HTTPie/0.8.0
Connection: keep-alive
Content-Type: application/json; charset=utf-8
{"username": "abhishek", "email": "john#doe.com", "password": "aaezaakmi1"}
HTTP/1.0 400 BAD REQUEST
Date: Sat, 24 Jan 2015 09:40:03 GMT
Server: WSGIServer/0.1 Python/2.7.9
Vary: Accept, Cookie
Content-Type: application/json
Allow: POST, OPTIONS
{"detail":"JSON parse error - Expecting property name enclosed in double quotes: line 1 column 2 (char 1)"}
The problem that you are running into is that your request is already being parsed, and you are trying to parse it a second time.
From "How the parser is determined"
The set of valid parsers for a view is always defined as a list of classes. When request.data is accessed, REST framework will examine the Content-Type header on the incoming request, and determine which parser to use to parse the request content.
In your code you are accessing request.DATA, which is the 2.4.x equaivalent of request.data. So your request is being parsed as soon as you call that, and request.DATA is actually returning the dictionary that you were expecting to parse.
json = request.DATA
is really all you need to parse the incoming JSON data. You were really passing a Python dictionary into json.loads, which does not appear to be able to parse it, and that is why you were getting your error.
I arrived at this post via Google for
"detail": "JSON parse error - Expecting property name enclosed in double-quotes":
Turns out you CANNOT have a trailing comma in JSON.
So if you are getting this error you may need to change a post like this:
{
"username" : "abhishek",
"email" : "john#doe.com",
"password" : "secretpass",
}
to this:
{
"username" : "abhishek",
"email" : "john#doe.com",
"password" : "secretpass"
}
Note the removed comma after the last property in the JSON object.
Basically, whenever you are trying to make a post request with requests lib, This library also contains json argument which is ignored in the case when data argument is set to files or data. So basically when json argument is set with json data. Headers are set asContent-Type: application/json. Json argument basically encodes data sends into a json format. So that at DRF particularly is able to parse json data. Else in case of only data argument it is been treated as form-encoded
requests.post(url, json={"key1":"value1"})
you can find more here request.post complicated post methods