Scala HTTP4s print entire HTTP Error response - json

I wrote this code to use the http4s client library to make a POST call to a REST web service
val client = SimpleHttp1Client()
val form = UrlForm("username" -> userName, "password" -> password)
val uri = Uri.fromString(url).valueOr(throw _)
val list = List(`Content-Type`(MediaType.`application/json`), Accept(MediaType.`application/json`))
val req = POST(uri, form).map(_.putHeaders(list :_*))
try {
val result = client.expect[String](req).unsafePerformSync
Some(result)
} catch {
case e : Throwable => println(e.getMessage); None
}
Right now, the code gets a 400 Bad Request error and I have no idea about why. I am not able to print the entire body of the error in the code above.
when I make the same REST call thru POSTMAN I can see 400 Bad request error body
{
"errors": [
"password: field is required",
"username: field is required"
]
}
How can I print the entire error body in my code?
Also, in postman when I set Content-Type and Accept headers the post call succeeds. In the code above, I am setting the same 2 headers and the same json body but still this code gets 400 error.
val list = List[Header](Header("Accept", "application/json"), Header("Content-Type", "application/json"))
val req = POST(uri, form).map(_.replaceAllHeaders(list :_*))

I had a conversation with the http4s team on gitter and I found the answer. since that conversation is not indexed by google I am writing the answer here
val output : Either[String, Foo] = client.fetch(request) {
case Successful(resp) => resp.as[Foo].map(Right(_))
case resp => resp.as[String].map(Left(_))
}
This if there is an error. we get the entire body of the error with this.

Related

AWS API Gateway Malformed Lambda Response

The documentation states that the json should return containing a body,headers,and a status code all of which I have. However for whatever reason when I test it in API gateway it returns a malformed response.
This is the output of the method below it.
"{\"body\": 200, \"headers\": {\"Content-type\":
\"application/json\"}, \"statusCode\": 200}"
def addnumbers(message, context):
result = message['num1'] + 1
print(result)
resp = {
"statusCode": 200,
"body": result,
"headers": { "Content-type": "application/json"}
}
return (json.dumps(resp))
I am currently passing in num1=1 and it doesn't give any better error message. Any guidance would be appreciated.
Ok buckle in for an answer.
Make sure you have proxy integration enabled on whatever resource you want in your API.
Now go to your lambda. Look at how I was previously trying to pass in num1.I was trying to get it from the "Event" or message. This is where I was tripping up. Also note (you can't do a get with a body)
Rather the input to the lambda should be like this.
{
"queryStringParameters": {
"input": "Whatever the input is you want the lambda to test"
}
}
So now that we have our test configured for the lambda we need to code the lambda itself.
I put this code within :
def lambda_handler(event, context):
number = "Hello, " + event['queryStringParameters']['input']
out = {}
out['statusCode'] = 200
out['body'] = number
return (out)
Now if you test it should be fine.
Go back to the API Gateway
In the "Query Strings" Section put in input=randomname
It should now return with hello, randomname

Play Framework 2.5 ajax json route parameter async MongoDB

POST ing json from javascript to server in Play Framework:
var myJson = {"name": "joe", "age":20};
var obj = JSON.parse(myJson);
$.ajax(jsRoutes.controllers.MyController.create(obj));
Now, I have the javascript router configured fine. If i recieve the obj as a string I can print it out to the console just fine.
routes.conf:
POST /person/add controllers.MyController.createFromAjax(ajax: String)
BUT, I want to write the json to MongoDB using an Async promise which Activator gives the compile time error:
scala.concurrent.Future[play.api.mvc.Result][error] cannot be applied to (String)
I have other routes that take no parameters that receive json using Postman and write it to MongoDB just fine
routes.conf
POST /heartrates/bulk controllers.HRController.createFromJson
If I omit the parameter on the route that receives the json from Ajax instead of using Postman I get a HTTP 400 error in the browser.
POST http://localhost:9000/person/add 400 (Bad Request)
SO, my question is, Ajax needs a parameter but String wont work. Play documentation says json is always received as a String. What am I doing wrong here?
Scala Controller Code taken from Lightbend seed Play.Reactive.MongoDB:
def createBulkFromAjax = Action.async(parse.json) { request =>
val documents = for {
heartRate <- request.body.asOpt[JsArray].toStream
maybeHeartRate <- heartRate.value
validHeartRate <- maybeHeartRate.transform(transformer).asOpt.toList
} yield validHeartRate
for {
heartRate <- hrFuture
multiResult <- heartRate.bulkInsert(documents = documents, ordered = true)
} yield {
Logger.debug(s"Successfully inserted with multiResult: $multiResult")
Created(s"Created ${multiResult.n} heartRate")
}
}
I think you're getting mixed up between the parameters you pass to your Action as part of the jsRoutes call, and parameters that get passed to endpoints (i.e. the query string, query parameters etc).
Play will return a 400 Bad Request if you've declared a non-optional parameter (like you did with ajax: String) and you don't then actually supply it in your request.
While conceptually you are passing obj to your action, it's not as a query parameter - you've declared that your endpoint expects an HTTP POST - so the JSON should be in the HTTP request body. Notice your other endpoints don't take any query parameters.
So step 1 is to fix your routes file (I've renamed your method to match your other existing working one):
POST /person/add controllers.MyController.createFromJson
If you look at the Play documentation for the Javascript reverse router, you'll see that you'll need to set the type (aka HTTP method) if you're doing something other than a GET. So, step 2, here's what your Javascript should look like to achieve a POST:
var myJson = {"name": "joe", "age":20};
var obj = JSON.stringify(myJson);
var r = controllers.MyController.createFromJson;
$.ajax({url: r.url, type: r.type, data: obj });
After those changes you should be good; your controller code looks fine. If you still get 400 Bad Request responses, check that jQuery is setting your Content-Type header correctly - you may need to use the contentType option in the jQuery $.ajax call.
Edit after still getting 400 errors:
I've just noticed that you were using JSON.parse in your Javascript - as per this answer you should be using JSON.stringify to convert an object into something jQuery can send - otherwise it may try to URLEncode the data and/or send the fields as query parameters.
The other thing to look at is whether the JSON you are sending actually agrees with what you're trying to parse it as. I'm not sure if you've provided a simplified version for this question but it looks like you're trying to parse:
{"name": "joe", "age":20}
Using:
request.body.asOpt[JsArray]
Which will always result in a None - you didn't give it an array.
The Answer to ajax javascript routes in Play Framework 2.5 for ReativeMongo:
routes.conf:
GET /javascriptRoutes controllers.HRController.javascriptRoutes
HRController:
def javascriptRoutes = Action { implicit request =>
Ok(
JavaScriptReverseRouter("jsRoutes")(
routes.javascript.HRController.createBulkFromAjax
)
).as("text/javascript")
}
routes.conf:
POST /heartrates/add controllers.HRController.createBulkFromAjax
main.scala.html:
<script type="text/javascript" src="#routes.HRController.javascriptRoutes"></script>
javascript:
var r = jsRoutes.controllers.HRController.createBulkFromAjax();
$.ajax({url: r.url, type: r.type, contentType: "application/json", data: JsonString });
HRController:
def createBulkFromAjax = Action.async(parse.json) { request =>
//Transformation silent in case of failures.
val documents = for {
heartRate <- request.body.asOpt[JsArray].toStream
maybeHeartRate <- heartRate.value
validHeartRate <- maybeHeartRate.transform(transformer).asOpt.toList
} yield validHeartRate
for {
heartRate <- hrFuture
multiResult <- heartRate.bulkInsert(documents = documents, ordered = true)
} yield {
Logger.debug(s"Successfully inserted with multiResult: $multiResult")
Created(s"Created ${multiResult.n} heartRate")
}
}
HRController.createBulkFromAjax was built from a Lightbend activator ui seed example called play.ReactiveMogno

How to test a play framework controller that uses a custom bodyparser?

I'm trying to test a play framework 2.5.10 controller that uses a custom bodyparser and JSON validator:
class MessagingController {
def validateJson[A : Reads] = parse.json.validate(
_.validate[A].asEither.left.map(e => BadRequest(JsError.toJson(e)))
)
def createMessageThread() = Action(validateJson[InboundMessageThread]) { request =>
Ok("OK") }
}
When I run a simple test case on this I get an error:
For request 'POST /api/v1/messageThreads' [Invalid Json: No content to map due to
end-of-input at [Source: akka.util.ByteIterator$ByteArrayIterator$$anon$1#5693d1d2; line: 1, column: 0]]
The test is:
val fakeRequest = FakeRequest()
.withHeaders(HeaderNames.CONTENT_TYPE -> "application/json")
.withBody(Json.parse(
s"""
|{
| "participants": [
| {"id": $currentUserId, "isAdmin": false}
| ],
| "isGroupThread": false
|}
""".stripMargin))
val result = messagingController.createMessageThread()(fakeRequest).run()
status(result) mustBe OK
If I change the controller's action to just Action and then validate the JSON in the body of the controller it works. I don't want to do that though because lots of my REST controllers need to parse JSON and using validateJson[T] reduces boilerplate. The code also works if I submit the same payload using curl.
How can I test this controller?
The following sort of request works:
val request = FakeRequest()
.withHeaders(CONTENT_TYPE -> "application/json")
.withBody(Json.fromJson[Place](body).get)
Try this, hope this helps
val postRequest = FakeRequest(POST, "/route/to/hit").withJsonBody(Json.obj(
"participants" -> Json.arr(Json.obj("id" -> "currentUserId", "isAdmin" -> false)),
"isGroupThread" -> false
))
val result = await(messagingController.createMessageThread()(postRequest))
status(result) must be equalTo (FORBIDDEN)
Rather than calling the Action with a JSON bodyparser, I'd suggest calling it with an ActionBuilder using an AnyContent body parser and then calling Json.validate from inside the body -- from there, you get a stack trace and see what's failed:
class MessagingController {
def createMessageThread() = Action { request =>
request.json.validate[MessageThread] {
case JsError(e) =>
logger.error(s"Failure parsing because $e")
BadRequest(JsError.toJson(e))
case JsSuccess(e) =>
Ok("OK")
}
}
}

AWS Lambda function - can't call update thing shadow

According to boto3 documentation here: https://boto3.readthedocs.org/en/latest/reference/services/iot-data.html#client the update_thing_shadow method takes the thingName & JSON payload as parameters. Currently it reads:
client = boto3.client('iot-data', region_name='us-east-1')
data = {"state" : { "desired" : { "switch" : "on" }}}
mypayload = json.dumps(data)
response = client.update_thing_shadow(
thingName = 'MyDevice',
payload = b'mypayload'
)
When I use the command line there's no problem but can't seem to get it right from within the lamba function. I've called it with numerous versions of code (json.JSONEncoder, bytearray(), etc..) without any luck. The errors range from syntax to (ForbiddenException) when calling the UpdateThingShadow operation: Bad Request: ClientError. Has anyone had success calling this or a similar method from within a AWS lambda function? Thanks.
This code is working fine for me:
def set_thing_state(thingName, state):
# Change topic, qos and payload
payload = json.dumps({'state': { 'desired': { 'property': state } }})
logger.info("IOT update, thingName:"+thingName+", payload:"+payload)
#payload = {'state': { 'desired': { 'property': state } }}
response = client.update_thing_shadow(
thingName = thingName,
payload = payload
)
logger.info("IOT response: " + str(response))
logger.info("Body:"+response['payload'].read())
def get_thing_state(thingName):
response = client.get_thing_shadow(thingName=thingName)
streamingBody = response["payload"]
jsonState = json.loads(streamingBody.read())
print jsonState
#print jsonState["state"]["reported"]
Good luck
garnaat is right.
Just replace payload = b'mypayload' with payload = mypayload and it should work.

Tornado POST request not detecting json input as argument

I have written a service which takes a json as input. I am using the website hurl.it to send post requests to check. Below is my code snippet:
class BatchSemanticSimilarityHandler(tornado.web.RequestHandler):
def post(self):
self.set_header('Access-Control-Allow-Origin', '*')
self.set_header('Access-Control-Allow-Credentials', 'true')
self.set_header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS')
self.set_header('Access-Control-Allow-Headers','Origin, Accept, Content-Type, X-Requested-With, X-CSRF-Token')
data = json.loads(self.request.body)
apikey = data["apikey"]
try:
UA = self.request.headers["User-Agent"]
except:
UA = "NA"
if bool(usercoll.find_one({"apikey":apikey})) == True:
sentence = data["sentence"]
sentence_array = data["sentence_array"]
n = data["num_of_results"]
if sentence is None or sentence_array is [] or apikey is None or n is None:
self.set_status(200)
output = {"error":[{"code":334,"message":"Bad Input data"}]}
misscoll.insert({"apitype":"batchsemanticsimilarity","timestamp":datetime.datetime.now(), "ip":self.request.remote_ip, "useragent":UA, "uri":self.request.uri,"apikey":apikey, "output":output, "input":{"s1":sentence,"s2":sentence_array}})
self.write(output)
return
results = nb.get_similar(sentence, sentence_array, apikey, n)
print "results is",results
output = {"similar_sentences": results, 'credits':'ParallelDots'}
hitscoll.insert({"apitype":"batchsemanticsimilarity","timestamp":datetime.datetime.now(), "ip":self.request.remote_ip, "useragent":UA, "uri":self.request.uri,"apikey":apikey, "output":output, "input":{"s1":sentence,"s2":sentence_array}})
self.write(output)
return
else:
rejectcoll.insert({"apitype":"batchsemanticsimilarity","apikey":apikey,"timestamp":datetime.datetime.now(), "ip":self.request.remote_ip, "useragent":UA, "url":self.request.uri})
self.write({"error":[{"code":333,"message": "Bad Authentication data"}]})
return
The json that I am giving as the body of the request is as below:
{
"sentence": "BJP leads in Bengaluru civic body`s poll, all eyes on JD(S)",
"sentence_array": [
"Narendra Modi is the prime minister",
"Sonia Gandhi runs Congress",
"Sachin is a good batsman"
],
"apikey": "DyMe1gSNhvMV1I1b20a7KARYIwuQX5GAQ",
"num_of_results": 2
}
I have verified on jsonlint that this is a valid JSON.
However while sending the request it gives me below error:
ValueError: No JSON object could be decoded
Can anyone please help me sort this out!!
The JSON object that you are passing in POST request is encoded into the url.
JSON library cannot read the encoded data.So you need to decode the url first.
Decoding of url can be done using urlparse library in python.so you need something like this.
post_data=urlparse.parse_qsl(self.request.body)
According to your need of final format to read there are various methods in urlparse.check this
or
As specified in the docs you can override a method to enable JSON parsing
def prepare(self):
if self.request.headers["Content-Type"].startswith("application/json"):
self.json_args = json.loads(self.request.body)
else:
self.json_args = None
check this