How to convert request data to json in AWS Lambda services? - json

"error%5Bcode%5D=BAD_REQUEST_ERROR&error%5Bdescription%5D=Payment+failed&error%5Bsource%5D=gateway&error%5Bstep%5D=payment_authorization&error%5Breason%5D=payment_failed&error%5Bmetadata%5D=%7B%22payment_id%22%3A%22pay_Es97gMGzx61l1u%22%2C%22order_id%22%3A%22order_Es96Rxp5OmnVVF%22%7D"
We are currently migrating to Lambda services from Flask. In Flask I was able to get the data in dictionary but in the AWS Lambda services, I am receiving the data as string, does anyone know how to parse this or convert it into a json or dictionary?
Thanks for ur time (:

This example string looks like an encode URI String. Where do you get it ? Could you provide more informations about usage context : API Gateway or request from another lambda, and how do you get this this?
With Pyhton usually you can get the path parameters and the query string parameters as a dictionnary with :
def my_handler(event, context):
params = event["pathParameters"]
query = event["queryStringParameters"]

Related

Unescape JSON from terraform provider

I am developing a custom terraform provider in Go.
I have a JSON parameter like this:
local_context_data = jsonencode({"hello"="world"})
When I check what is sent to the API with TCPdump I can see that the JSON is escaping like this:
{\"hello\":\"world\"}.
But it generates an error because the API is waiting for real JSON without escape.
Do we have a function in Go to unescape this?
Or better: do we have a function in the terraform SDK?
if your API expects json, what you want is probably
local_context_data = {"hello"="world"}
You only need jsonencode if you want to send a json string within the json.

Spark Read JSON with Request Parameters

I'm trying to read a JSON response from IBM Cloud's DB2 Warehouse documentation. This requires me to pass a request body wherein I have to supply userid and password as request parameters.
To read using spark.read.json, I did not find anything wherein request parameters could be supplied. Is there anyway using which we could do that?
Usually I would read the JSON using Scala alone using scalaj-http and play-json libraries like:
val body = Json.obj(Constants.KEY_USERID -> userid, Constants.KEY_PASSWORD -> password)
val response = Json.parse(Http(url + Constants.KEY_ENDPOINT_AUTH_TOKENS)
.header(Constants.KEY_CONTENT_TYPE , "application/json")
.header(Constants.KEY_ACCEPT , "application/json")
.postData(body.toString())
.asString.body)
My requirement is I cannot use these 2 libraries and have to do it using scala with the spark framework.
You can not use spark.read.json directly for REST API data ingestion.
First, make your API call request to get response data and then convert it to DataFrame with Spark. Note that if your API is paginated then, you'll need to make multiple calls to get all data.
For your example, you need to call authentication endpoint in order to get a Bearer token and then add it to the request header :
Authorization: Bearer <your_token>
All this part could be done using only Scala (example scala.io.Source.fromURL).
Once you get the response_data, use spark to convert it to DF :
import spark.implicits._
val df = spark.read.json(Seq(response_data).toDS)

What's the proper way to extract data from a google protobuff object?

I have never encountered this sort of collection or object before until now (its the response from a request to Google-Cloud-Vision API).
I wrote a class that uses the API and does what I want correctly. However the only way that I can extract/manipulate data in the response is by using this module:
from google.protobuf.json_format import MessageToJson
I basically serialized the protobuff into a string and then used regex to get the data that I want.
There MUST be a better way than this. Any suggestions? I was hoping to have the API response give me a json dict or json dict of dicts etc... All I could come up with was turning the response into a string though.
Here is the file from the github repository:
image_analyzer.py
Thank you all in advance.
The built in json module will parse the string into a dictionary, like json.loads(MessageToJson(response1)).
You can just access the fields in the message object directly, e.g.:
response1 = vision_client.face_detection(image=image)
print(response1)
print(response1.face_annotations[0].detection_confidence)

Google Cloud Endpoints allowed method's return types

I'm developing a REST API using Cloud Endpoints. I know that as per the documentation each API method should return an entity that is then automatically converted to a valid JSON string.
However, I'm dealing with an authentication library that in some cases returns a JSON which should be passed back to the client as a response.
Sticking with the default approach, meaning returning an entity, would still be possible, but it would involve a number of obnoxious intermediate steps, like parsing the JSON and filling the right fields of the entity to be returned according to the JSON content.
I was wondering if there is a more straightforward way to instruct the API to directly return the JSON string, instead of converting it to an entity just to have it translated back to the source JSON.
One thing that probably may help you.
Suppose you have a RPC response class as
from protorpc import messages
from protorpc import message_types
from protorpc import remote
class ApiResponse(message.Messages):
response = messages.StringField(1, required = True)
You can return your JSON in response field by using JSON module. First of all import json in your endpoints api
import json
Suppose your response is
json_response = [{'name': 'Jack', 'age':12}, {'name': 'Joe', 'age':13}]
In your response to this api you can do this:-
return ApiResponse(response = json.dumps({'data': json_response}))
json.dumps() converts your dictionary object into JSON string that can be passed to response of ApiResponse class.
After you receive response in client side(javascript), you can simply parse it to JSON using
JSON.parse(response)

retrieve json from google app engine datastore

I have stored some simple data in the GAE datastore. Now I'd like to pull it out, but I'd like to do so with the results as JSON. Is there a simple way to do this?
You can first convert your datastore model to a dictionary and then use simplejson (python 2.5) or json (python 2.7) package to convert the dictionary to json. Typically this is how the last line of your handler will look like:
self.response.out.write(simplejson.dumps(some_datastore_entity.to_dict()))
The new ndb interface to the datastore provides a to_dict method by default. You can check it out here
class Handler(webapp2.RequestHandler):
def get(self):
<do your GQLQuery here>
self.response.headers['Content-Type'] = 'application/json'
self.response.body = json.dumps(<your data in dict or dict list>)
self.response.set_status(200)