Formatting DynamoDB data to normal JSON in API gateway - json

just like the question asked here
I do a query or scan operation on dynamoDB using dynamoDB proxy service on AWS API Gateway to read data for the Client and I get DynamoDB formatted JSON data in reply.
Although I can use the "Method Response" to convert but when the Data is above 1000 records - I cannot handle it due to the limitation of foreach loop in Method Response.
Is there a flag or a setting somewhere in dynamodb or in api gateway such that I get normal json rather than the dynamoDB formatted JSON ?
DynamoDB Formatted JSON example
{
"videos": [
{
"file": {
"S": "file1.mp4"
},
"id": {
"S": "1"
},
"canvas": {
"S": "This is Canvas1"
}
},
{
"file": {
"S": "main.mp4"
},
"id": {
"S": "0"
},
"canvas": {
"S": "this is a canvas"
}
}
]
}
Normal Json example of the same
{
"videos": [
{
"file": "file1.mp4",
"id": "1",
"canvas": "This is Canvas1"
},
{
"file": main.mp4",
"id": "0",
"canvas": "this is a canvas"
}
]
}

Using an Integration Response mapper!
The mapper would probably look something like: (I'm using your provided response)
#set($inputRoot = $input.path('$'))
{
"videos": [
#foreach($elem in $inputRoot.Items)
{ "file": "$elem.file.S",
"id": "$elem.id.S",
"canvas": "$elem.canvas.S" }#if($foreach.hasNext), #end
#end
]
}
Please consult the linked documentation for more complete guidance on attaching AWS Services directly to API Gateway.

The possible solution for this is using a aws lambda to query/scan dynamoDb and reply back the domain object after converting it into a json using some json lib, like GSON or jackson.
If any one has knowledge about an alternate option - please do post here. Thanks

Related

Apache VTL - Copy node

Is there a way to do deep copy using apache VTL?
I was trying to use x-amazon-apigateway-integration using requestTemplates.
The input JSON is as shown below,
{
"userid": "21d6523137f6",
"time": "2020-06-16T15:22:33Z",
"item": {
"UserID" : { "S": "21d6523137f6" },
<... some complex json nodes here ...>,
"TimeUTC" : { "S": "2020-06-16T15:22:33Z" },
}
}
The requestTemplate is as shown below,
requestTemplates:
application/json: !Sub
- |
#set($inputRoot = $input.path('$'))
{
"TableName": "${tableName}",
"ConditionExpression": "attribute_not_exists(TimeUTC) OR TimeUTC > :sk",
"ExpressionAttributeValues": {
":sk":{
"S": "$util.escapeJavaScript($input.path('$.time'))"
}
},
"Item": "$input.path('$.item')", <== Copy the entire item over to Item.
"ReturnValues": "ALL_OLD",
"ReturnConsumedCapacity": "INDEXES",
"ReturnItemCollectionMetrics": "SIZE"
}
- {
tableName: !Ref EventsTable
}
The problem is, the item gets copied like,
"Item": "{UserID={S=21d6523137f6}, Lat={S=37.33180957}, Lng={S=-122.03053391}, ... other json elements..., TimeUTC={S=2020-06-16T15:22:33Z}}",
As you can see, the whole nested json become a single atribute. While I expected it to become a fully blown json node on its own like below,
"Item": {
"UserID" : { "S": "21d6523137f6" },
"Lat": { "S": "37.33180957" },
"Lng": { "S": "-122.03053391" },
<.... JSON nodes ...>
"TimeUTC" : { "S": "2020-06-20T15:22:33Z" }
},
Is it possible to deep/nested copy operation on a json node like above without doing the kung-fu of iterating the node and appending the childs o a json node variable etc...
btw, I'm using AWS API Gateway request template, so it may not support all the Apache VTL templating options.
You need to use the $input.json method instead of $input.path.
"Item": $input.json('$.item'),
Note that I removed the double quotes.
If you had the double quotes because you want to stringify $.item, you can do that like so:
"Item": "$util.escapeJavaScript($input.json('$.item'))",

Go dialogflow sdk WebhookResponse doesn't return the correct json

I'm trying to write a webhook in Go for Dialogflow, I'm using the apiv2 of the official SDK
google.golang.org/genproto/googleapis/cloud/dialogflow/v2
But I can't generate a correct response using the official sdk.
What I mean is that following the documentation and the WebhookResponse struct I can't generate the expected json for the response.
This is the piece of code that I'm using:
response = dialogflow.WebhookResponse{
FulfillmentMessages: []*dialogflow.Intent_Message{
{
Message: &dialogflow.Intent_Message_Card_{
Card: &dialogflow.Intent_Message_Card{
Title: "Title",
Subtitle: "Subtitle",
ImageUri: "https://example.com/images/example.png",
Buttons: []*dialogflow.Intent_Message_Card_Button{
{
Text: "Button",
Postback: "https://example.com/path/for/end-user/to/follow",
},
},
},
},
},
},
}
This is the json that it generates:
{
"fulfillment_messages": [
{
"Message": {
"Card": {
"title": "Title",
"subtitle": "Subtitle",
"image_uri": "https://example.com/images/example.png",
"buttons": [
{
"text": "Button",
"postback": "https://example.com/path/for/end-user/to/follow"
}
]
}
}
}
]
}
But this is the json that I should send back (according to the official documentation)
"fulfillmentMessages": [
{
"card": {
"title": "card title",
"subtitle": "card text",
"imageUri": "https://example.com/images/example.png",
"buttons": [
{
"text": "button text",
"postback": "https://example.com/path/for/end-user/to/follow"
}
]
}
}
]
}
So my json doesn't work, because it has the Message that shouldn't be there, and Card with uppercase first letter. I've tried to send the json of the documentation and it works, Dialog Flow responds correctly.
I don't understand how to generate the correct json using the official SDK. Please consider that I'm pretty new using Go Lang. This is my first project.
This is the documentation that I'm using at the moment:
https://pkg.go.dev/google.golang.org/genproto/googleapis/cloud/dialogflow/v2?tab=doc#WebhookResponse
As you can see the FulfillmentMessages is an array of Intent_Message
FulfillmentMessages []*Intent_Message
And the Intent_Message has to contain Message (here the documentation)
Thanks in advance for any help and suggestions.
H2K
UPDATE:
if I use log.Println(response) I can see the correct response inside the log
fulfillment_messages:{card:{title:"Title" subtitle:"Subtitle" image_uri:"https://example.com/images/example.png" buttons:{text:"Button" postback:"https://example.com/path/for/end-user/to/follow"}}}
It is not a JSON but the structure is correct, no Message, no Card...
So the problem is when I return it with Gin and the command:
c.JSON(200, response)
I've found a solution!
I need to use the jsonpb marshaler and return it as string with Gin
Here an example:
m := jsonpb.Marshaler{}
result, _ := m.MarshalToString(response)
c.String(200, result)
I've totally went crazy on it, I hope that it could be helpful for someone else.

Reading complex json data without iteration

I am working with some data and often the data is nested and i am required to perform some CRUD operations based on the structure of the data i have. For instance i have this json structure
{
"_id": "KnNLkJEhrDsvWedLu",
"createdAt": {
"$date": "2016-10-13T11:24:13.843Z"
},
"services": {
"password": {
"bcrypt": "$2a$30$1/cniPwPNCuwZ/MQDPQkLej..cAATkoGX.qD1TS4iHgf/pwZYE.j."
},
"email": {
"verificationTokens": [
{
"token": "qxe_T9IS7jW7gntpK0Q7UQ35RJ9jO9m2lclnokO3z87",
"address": "drwho#gmail.com",
"when": {
"$date": "2016-10-13T11:24:14.428Z"
}
}
]
},
"resume": {
"loginTokens": []
}
},
"username": "doctorwho",
"emails": [
{
"address": "drwho#gmail.com",
"verified": false
}
],
"persodata": {
"lastlogin": {
"$date": "2016-10-13T11:29:36.816Z"
},
"fname": "Doctor",
"lname": "Who",
"mobile": "+4480000000",
"identity": "1",
"email": "drwho#gmail.com",
"gender": null
}
}
I have several data sets with such complex structure. I need to read the data, edit and also delete. Before i get to iteration, i was wondering how i can read the data without iteration then iterate when i absolutely have to.
What are the rules i should keep in mind when reading such complex json structures to enable me read any complex structure i come across?.
I am currently using javascript but i am looking for rules that apply in other languages as well.
Parsing Json in JavaScript should be easy. http://www.json.org/js.html.
"Since JSON is a proper subset of JavaScript, the compiler will correctly parse the text and produce an object structure". Just follow the examples on that page.
If you want to use another language, in Java you could use Jackson or Gson to map those json strings to objects. Then using them becomes easy. Both libraries are annotation based, and wouldn't be difficult to implement.

REST-request yields error because Jira thinks that required field is missing

I'm trying to create a Jira-issue via REST. My request looks like this:
Method: POST
Content-Type: application/json
Body: '{"fields":{"project":"ID"}}'
The response I get looks like this
{"errorMessages":[],"errors":{"project":"project is required"}}
which is strange, since I'm providing a project in my request. Does anyone see what I'm missing here?
Seems like you are sending an incorrect JSON to JIRA, according to JIRA documentation an issue is formed like
"fields": {
"project": {
"id": "10000"
},
"summary": "something's wrong",
"issuetype": {
"id": "10000"
},
"assignee": {
"name": "homer"
}
}
but you are sending
{
"fields": {
"project": "ID"
}
}

How can I know the data type in graph API response?

I'm beginner of graph API.
For my Application, I have to use many graph API at the same time.
My question is... how can I know the response's data type?
=> How can I know the response that it is the friends data?
ex) for friend data.
In response data, I can't find the data name for friend.(== like friend tag name in XML response)
request Graph URL
= https://graph.facebook.com/100000726819009/friends
response data in graph API for friends
{
"data": [
{
"name": "Jong-Mu Choi",
"id": "1931865"
},
{
"name": "Joo Yeon Kim",
"id": "4812863"
},
{
"name": "Tommy Kang",
"id": "516573995"
},
{
"name": "Hyeoniee Jeong",
"id": "526059737"
},
{
"name": "Sung Park",
"id": "528812415"
}
]
}
http://graph.facebook.com/cocacola?metadata=1
You will see type=page at the bottom.