Lambda Edge 502 with custom response from viewer response - json

I'm using a URL query string to debug my viewer-request and viewer-response lambda#edge functions by returning the event as JSON to the frontend (FYI so I can check for the presence/absence of certain things via an external monitoring tool).
This works fine with the viewer-request: if I go to https://example.org/?debug_viewer_request_event I get a JSON of the viewer-request event:
import json
def lambda_handler(event, context):
request = event["Records"][0]["cf"]["request"]
if "debug_viewer_request_event" in request["querystring"]:
response = {
"status": "200",
"statusDescription": "OK",
"headers": {
"cache-control": [
{
"key": "Cache-Control",
"value": "no-cache"
}
],
"content-type": [
{
"key": "Content-Type",
"value": "application/json"
}
]
},
"body": json.dumps(event)
}
return response
# rest of viewer-request logic...
Testing with cURL:
curl -i https://example.org/?debug_viewer_request_event
HTTP/2 200
content-type: application/json
content-length: 854
server: CloudFront
date: Mon, 26 Apr 2021 06:05:28 GMT
cache-control: no-cache
x-cache: LambdaGeneratedResponse from cloudfront
via: 1.1 xxxxxxxxxxx.cloudfront.net (CloudFront)
x-amz-cf-pop: AMS50-C1
x-amz-cf-id: pU0ItvQA1-r5v3yR1Dl6Z3VpPW_EuuUCHhnOD60uLhng...
{"Records": [{"cf": {"config": {"distributionDomainName": "xxxxxxx.cloudfront.net", "distributionId": "xxxxxxx", "eventType": "viewer-request", "requestId": "pU0ItvQA1-r5v3yR1Dl6Z3VpPW_EuuUCHhnOD60uLhng...
However when I do the same with the viewer-response I get a 502 error:
the code is the same except debug_viewer_request_event is debug_viewer_response_event
if I don't include the debug query string, the response is 200 OK so I know overall both lambdas are working properly (with the exception of the debug for the viewer-response)
Here is the cURL output:
curl -i https://example.org/?debug_viewer_response_event
HTTP/2 502
content-type: text/html
content-length: 1013
server: CloudFront
date: Mon, 26 Apr 2021 06:07:39 GMT
x-cache: LambdaValidationError from cloudfront
via: 1.1 xxxxxxxxx.cloudfront.net (CloudFront)
x-amz-cf-pop: AMS50-C1
x-amz-cf-id: NqXQ-FFEsIX-fEt8IvlHFTYoQdrZSGPScq1H-KNwVWR0-xxxxxx
The Lambda function result failed validation: The function tried to add, delete, or change a read-only header
If I look at the docs, the list of "Read-only Headers for CloudFront Viewer Response Events" is:
Content-Encoding
Content-Length
Transfer-Encoding
Warning
Via
As far as I can see I'm not directly changing any of these headers, but I'm guessing because I'm modifying the response, headers such as Content-Length are modified
Q: Is there a way to return the viewer-response event as JSON to the frontend for debugging or is it simply not possible due to not being able to change Content-Length?

As far as I can see I'm not directly changing any of these headers,
but I'm guessing because I'm modifying the response, headers such as
Content-Length are modified
I agree, I think your issue is that you are returning the response instead of calling
callback(null, response);
where callback should be the third argument to your lambda handler func:
def lambda_handler(event, context, callback):
Since content-length is not mutable, we should assume (and I checked this is true in practice at least for viewer request functions), cloudfront will generate it for you when you generate a response in the edge function.

Related

MacOS regex grep getting value between curly brace

I'm trying to get a curl response between {}. I have found and tested a regex command that works files with Sublime or an online tester.
The problem occurs when I try to execute it with grep from MacOS. I have installed the grep from the brew library, but even though the installation occurred 100%, the command doesn't work. Deleting all break lines of file/response (performing debug), the command works! But in my case, the curl response comes with break lines, so I should be able to handle it.
Could someone tell me why it is occurring with MacOS and how I can solve it?
Curl response:
HTTP/2 401
www-authenticate: Digest realm="MMS Public API", domain="", nonce="8878t9jXCP7+", algorithm=MD5, qop="auth", stale=false
content-type: application/JSON
content-length: 106
x-envoy-upstream-service-time: 3
date: Fri, 13 Jan 2023 17:04:03 GMT
server: envoy
HTTP/2 400
date: Fri, 13 Jan 2023 17:04:04 GMT
strict-transport-security: max-age=31536000; include subdomains;
referrer-policy: strict-origin-when-cross-origin
x-permitted-cross-domain-policies: none
x-content-type-options: nosniff
content-type: application/json
x-frame-options: DENY
content-length: 200
x-envoy-upstream-service-time: 23
server: envoy
{
"detail": "Cluster asdasdasd cannot be created in a paused state.",
"error": 400,
"errorCode": "CANNOT_CREATE_PAUSED_CLUSTER",
"parameters" : [ "asdasdasd" ],
"reason": "Bad Request"
}
I want to get only the following lines:
{
"detail": "Cluster asdasdasd cannot be created in a paused state.",
"error": 400,
"errorCode": "CANNOT_CREATE_PAUSED_CLUSTER",
"parameters" : [ "asdasdasd" ],
"reason": "Bad Request"
}
My regexs:
{([\S\s]+)}
{[^{}]*}
Sublime response:
regextester.com result:
Better use jq.
Your input are plain JSON.
Example to retrieve errorCode:
curl ...... | jq '.errorCode'
jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.
Or gron
curl ...... | gron | awk -F' = ' '/^json.errorCode /{print $2}'
To install it:
go install github.com/tomnomnom/gron#latest
I have converted the curl command to python code. It is easier to handle with the response.
import requests
from requests.auth import HTTPDigestAuth
headers = {
'Content-Type': 'application/json',
}
params = {
'pretty': 'true',
}
json_data = {
'autoScaling': {
'diskGBEnabled': True,
},
'backupEnabled': False,
'paused': True,
'name': 'sdasdasd',
'providerSettings': {
'providerName': 'AWS',
'instanceSizeName': 'M10',
'regionName': 'US_EAST_2',
},
}
response = requests.post(
'https://cloud.mongodb.com/api/atlas/v1.0/groups/999999999999/clusters',
params=params,
headers=headers,
json=json_data,
auth=HTTPDigestAuth('user', 'password'),
)
print(response.content)

Google Cloud CDN serves stale content AFTER revalidation

I am attempting to use Google Cloud CDN with the stale-while-revalidate feature (Google Docs). However, even though I can see asynchronous revalidation requests being made to my backend server, the new content is never served by the CDN.
Here is an example of the response headers from the CDN:
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Age: 5202
Cache-Control: max-age=60, public, stale-while-revalidate=86400
Content-Length: 90
Content-Type: application/json
Date: Tue, 08 Feb 2022 16:39:09 GMT
Server: Development/1.0
Via: 1.1 google
You can see that the age is 5205 seconds even though the max-age is 60 sec.
If it has been more than 60 seconds since the last revalidation request, I will see the revalidation request in my server logs, but the content served from the CDN continues to be outdated on future requests.
Here is an example of the response headers directly from my backend server:
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: max-age=60, public, stale-while-revalidate=86400
Content-Length: 645
Content-Type: application/json
Date: Tue, 08 Feb 2022 18:09:39 GMT
Server: Development/1.0
And here is the Google Cloud Logging entry for the revalidation request. I believe it is showing that it received a 200 response as well:
{
"insertId": "1sxvbdng2vrqnhl",
"jsonPayload": {
"parentInsertId": "1sxvbdng2vrqng7",
"#type": "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry",
"statusDetails": "response_sent_by_backend",
"cacheId": "LAX-ba56a406"
},
"httpRequest": {
"requestMethod": "GET",
"requestUrl": "***masked request url***",
"requestSize": "1888",
"status": 200,
"responseSize": "1259",
"userAgent": "Cloud-CDN-Google (GFE/2.0)",
"remoteIp": "***masked request ip***",
"cacheLookup": true,
"cacheFillBytes": "1259",
"serverIp": "***masked request ip***",
"latency": "0.082249s"
},
"resource": {
"type": "http_load_balancer",
"labels": {
"backend_service_name": "test-be-service",
"target_proxy_name": "test-lb-target-proxy",
"project_id": "cdn-test-2-340220",
"zone": "global",
"url_map_name": "test-lb",
"forwarding_rule_name": "test-lb-forwarding-rule"
}
},
"timestamp": "2022-02-08T19:36:48.094836Z",
"severity": "INFO",
"logName": "projects/cdn-test-2-340220/logs/requests",
"trace": "projects/cdn-test-2-340220/traces/f6a372e62f789d61c3654e50111ffcb9",
"receiveTimestamp": "2022-02-08T19:36:48.360094896Z",
"spanId": "d20eb70cb8d7dff4"
}
This behavior is the same whether I activate stale-while-revalidate via the Cache-Control response header or the Cloud CDN configuration interface.
Either I am not understanding how this is supposed to work, or this is a bug in Google Cloud CDN. Any help resolving this would be greatly appreciated.

SoapUI vs Postman posting json

quite new to testing, but currently Im testing rest services...
Really like SoapUI (did some soap testing in the past), but I have problem with testing one service - it is service for creating an event - have tried it in postman(working) and in soapui(not working).
When I try to run it in SoapUI Im getting error:
Object reference not set to an instance of an object.
Here is code from postman request (getting status: 200 OK):
POST /Event.Api/v1/events HTTP/1.1
Host: 172.26.2.66
Content-Type: application/json
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJCT0hVU0xBViIsImZhbWlseV9uYW1lIjoiRE9LT1VQSUwiLCJQT0lEIjoiOGYyMTdiZWItZmQ3My00OThjLWFhZjktOWY0ZTk0YmRhMjIzIiwiZXhwIjoxNTE1MTUzNTg0fQ.2uB_pPXyl3wSAqonaDb5pLAwDb-BujMIU6Rdeg_73Jw
Cache-Control: no-cache
Postman-Token: 1d92a58e-c4de-f790-caca-fd983392a17e
{
"name": "Michalova událost",
"place": "AC",
"start": "2018-01-05T15:00:39.164Z",
"end": "2018-01-05T16:00:39.164Z",
"wholeDay": true,
"repeating": "NEOPAKUJE_SE",
"description": "Popis michalovy události"
}
edit: screen of soapUI request added
and here is request code from soapUI, dont undestand why, but there is not the json content I added in the post window
POST http://172.26.2.66/Event.Api/v1/events HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: application/json
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJCT0hVU0xBViIsImZhbWlseV9uYW1lIjoiRE9LT1VQSUwiLCJQT0lEIjoiZWJhMWVmNjMtNWUzNS00YTU3LTljNWMtNDY3ZmJhZmI5YTIyIiwiZXhwIjoxNTE1MTU4MTI3fQ.s5dNZxV6NME1G8rbYrsuv8sb1nMiB8z1GSVGHe3auVA
Content-Length: 220
Host: 172.26.2.66
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
and here is raw response from soapUI:
HTTP/1.1 500 Internal Server Error
Transfer-Encoding: chunked
Server: Kestrel
X-Powered-By: ASP.NET
Date: Fri, 05 Jan 2018 11:17:25 GMT
{
"errors": [
{
"code": "COMMON_ERROR",
"description": "Object reference not set to an instance of an object."
}
]
}
Anyone can help? Thanks for your replys.
P.S:think it is something with that json content, cause when Im testing get methods, everything is fine and Im getting expected results even in the soapUI
MJ

Drive API v2 & v3 (1.16) File Watch requests being denied when using Batch Request

I'm getting 403 watchDenied errors when I try to make File Watch requests in a Batch Request. Sending them normally is fine. I'm presuming that batching of File Watch requests is permitted?
Additional info: If I set an incorrect Address, I get the usually unpermitted webhook address error, and if I set a dummy file ID I get a 404, so it doesn't feel like the request I'm making is incorrect.
Example code (using the C# client library):
var request = new BatchRequest(service);
request.Queue<Channel>(service.Files.Watch(
new Channel
{
Id = [Guid being used],
Type = "web_hook",
Address = [notification endpoint],
Token = [token being used],
Expiration = DateTime.Now.AddHours(1).ToUnixTimeMilliseconds()
}, fileId)
, (content, error, i, message) =>
{
}
);
await request.ExecuteAsync();
API Request
Headers:
POST: https://www.googleapis.com/batch
User-Agent: OverDRIVE google-api-dotnet-client/1.16.0.0 (gzip)
Authorization: Bearer [token]
Content-Type: multipart/mixed; boundary="8d7a0653-4d9b-4c09-b701-9794341f882d"
Host: www.googleapis.com
Content-Length: 988
Accept-Encoding: gzip, deflate
Body:
--8d7a0653-4d9b-4c09-b701-9794341f882d
Content-Type: application/http
POST https://www.googleapis.com/drive/v2/files/[fileId]/watch
Content-Type: application/json; charset=utf-8
Content-Length: 241
{"address":"[notification endpoint]","expiration":1474036096103,"id":"938bd983-b071-4e63-8535-f20d7f39e248","token":"[token being used]","type":"web_hook"}
--8d7a0653-4d9b-4c09-b701-9794341f882d
Content-Type: application/http
POST https://www.googleapis.com/drive/v2/files/[fileId]/watch
Content-Type: application/json; charset=utf-8
Content-Length: 241
{"address":"[notification endpoint]","expiration":1474036096105,"id":"79303874-5f6c-49aa-a601-93950895ac0f","token":"[token being used]","type":"web_hook"}
--8d7a0653-4d9b-4c09-b701-9794341f882d--
Error example
{
"error": {
"errors": [
{
"domain": "push",
"reason": "watchDenied",
"message": "Watch request denied by backend"
}
],
"code": 403,
"message": "Watch request denied by backend"
}
}
Had an update from the developer support team for a ticket I submitted to them.
Unfortunately watch requests are known to largely incompatible with
Drive batch requests. Our Drive team recommended to make watch
requests separate from other requests.

DropWizard HealthCheck Json Response

I'm using dropwizard 0.6.2 for my service. The healthcheck response from dropwizard returns plain text. And I found a question in stackoverflow which had an answer that says we can pass a ObjectMapper to a healthcheck. But I couldn't able to find a way to pass the ObjectMapper to the HealthCheck.
Is there a way to return the healthcheck response in JSON?
As of Dropwizard 0.7, the /healthcheck path returns a JSON response:
HTTP/1.1 200 OK
Cache-Control: must-revalidate,no-cache,no-store
Content-Length: 299
Content-Type: application/json
Date: Thu, 14 Aug 2014 07:55:29 GMT
{
"My custom HealthCheck":
{
"healthy": true,
"message": "your message here"
},
"deadlocks":
{
"healthy": true
},
"storage":
{
"healthy": true
}
}
The Dropwizard use the codehale HealthCheck class.
You can call Result.healthy() and passing for parameter your JSON string.
In the method that you call the healthcheck you can use:
Result.healthy("your json");