MacOS regex grep getting value between curly brace - json

I'm trying to get a curl response between {}. I have found and tested a regex command that works files with Sublime or an online tester.
The problem occurs when I try to execute it with grep from MacOS. I have installed the grep from the brew library, but even though the installation occurred 100%, the command doesn't work. Deleting all break lines of file/response (performing debug), the command works! But in my case, the curl response comes with break lines, so I should be able to handle it.
Could someone tell me why it is occurring with MacOS and how I can solve it?
Curl response:
HTTP/2 401
www-authenticate: Digest realm="MMS Public API", domain="", nonce="8878t9jXCP7+", algorithm=MD5, qop="auth", stale=false
content-type: application/JSON
content-length: 106
x-envoy-upstream-service-time: 3
date: Fri, 13 Jan 2023 17:04:03 GMT
server: envoy
HTTP/2 400
date: Fri, 13 Jan 2023 17:04:04 GMT
strict-transport-security: max-age=31536000; include subdomains;
referrer-policy: strict-origin-when-cross-origin
x-permitted-cross-domain-policies: none
x-content-type-options: nosniff
content-type: application/json
x-frame-options: DENY
content-length: 200
x-envoy-upstream-service-time: 23
server: envoy
{
"detail": "Cluster asdasdasd cannot be created in a paused state.",
"error": 400,
"errorCode": "CANNOT_CREATE_PAUSED_CLUSTER",
"parameters" : [ "asdasdasd" ],
"reason": "Bad Request"
}
I want to get only the following lines:
{
"detail": "Cluster asdasdasd cannot be created in a paused state.",
"error": 400,
"errorCode": "CANNOT_CREATE_PAUSED_CLUSTER",
"parameters" : [ "asdasdasd" ],
"reason": "Bad Request"
}
My regexs:
{([\S\s]+)}
{[^{}]*}
Sublime response:
regextester.com result:

Better use jq.
Your input are plain JSON.
Example to retrieve errorCode:
curl ...... | jq '.errorCode'
jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.
Or gron
curl ...... | gron | awk -F' = ' '/^json.errorCode /{print $2}'
To install it:
go install github.com/tomnomnom/gron#latest

I have converted the curl command to python code. It is easier to handle with the response.
import requests
from requests.auth import HTTPDigestAuth
headers = {
'Content-Type': 'application/json',
}
params = {
'pretty': 'true',
}
json_data = {
'autoScaling': {
'diskGBEnabled': True,
},
'backupEnabled': False,
'paused': True,
'name': 'sdasdasd',
'providerSettings': {
'providerName': 'AWS',
'instanceSizeName': 'M10',
'regionName': 'US_EAST_2',
},
}
response = requests.post(
'https://cloud.mongodb.com/api/atlas/v1.0/groups/999999999999/clusters',
params=params,
headers=headers,
json=json_data,
auth=HTTPDigestAuth('user', 'password'),
)
print(response.content)

Related

Lambda Edge 502 with custom response from viewer response

I'm using a URL query string to debug my viewer-request and viewer-response lambda#edge functions by returning the event as JSON to the frontend (FYI so I can check for the presence/absence of certain things via an external monitoring tool).
This works fine with the viewer-request: if I go to https://example.org/?debug_viewer_request_event I get a JSON of the viewer-request event:
import json
def lambda_handler(event, context):
request = event["Records"][0]["cf"]["request"]
if "debug_viewer_request_event" in request["querystring"]:
response = {
"status": "200",
"statusDescription": "OK",
"headers": {
"cache-control": [
{
"key": "Cache-Control",
"value": "no-cache"
}
],
"content-type": [
{
"key": "Content-Type",
"value": "application/json"
}
]
},
"body": json.dumps(event)
}
return response
# rest of viewer-request logic...
Testing with cURL:
curl -i https://example.org/?debug_viewer_request_event
HTTP/2 200
content-type: application/json
content-length: 854
server: CloudFront
date: Mon, 26 Apr 2021 06:05:28 GMT
cache-control: no-cache
x-cache: LambdaGeneratedResponse from cloudfront
via: 1.1 xxxxxxxxxxx.cloudfront.net (CloudFront)
x-amz-cf-pop: AMS50-C1
x-amz-cf-id: pU0ItvQA1-r5v3yR1Dl6Z3VpPW_EuuUCHhnOD60uLhng...
{"Records": [{"cf": {"config": {"distributionDomainName": "xxxxxxx.cloudfront.net", "distributionId": "xxxxxxx", "eventType": "viewer-request", "requestId": "pU0ItvQA1-r5v3yR1Dl6Z3VpPW_EuuUCHhnOD60uLhng...
However when I do the same with the viewer-response I get a 502 error:
the code is the same except debug_viewer_request_event is debug_viewer_response_event
if I don't include the debug query string, the response is 200 OK so I know overall both lambdas are working properly (with the exception of the debug for the viewer-response)
Here is the cURL output:
curl -i https://example.org/?debug_viewer_response_event
HTTP/2 502
content-type: text/html
content-length: 1013
server: CloudFront
date: Mon, 26 Apr 2021 06:07:39 GMT
x-cache: LambdaValidationError from cloudfront
via: 1.1 xxxxxxxxx.cloudfront.net (CloudFront)
x-amz-cf-pop: AMS50-C1
x-amz-cf-id: NqXQ-FFEsIX-fEt8IvlHFTYoQdrZSGPScq1H-KNwVWR0-xxxxxx
The Lambda function result failed validation: The function tried to add, delete, or change a read-only header
If I look at the docs, the list of "Read-only Headers for CloudFront Viewer Response Events" is:
Content-Encoding
Content-Length
Transfer-Encoding
Warning
Via
As far as I can see I'm not directly changing any of these headers, but I'm guessing because I'm modifying the response, headers such as Content-Length are modified
Q: Is there a way to return the viewer-response event as JSON to the frontend for debugging or is it simply not possible due to not being able to change Content-Length?
As far as I can see I'm not directly changing any of these headers,
but I'm guessing because I'm modifying the response, headers such as
Content-Length are modified
I agree, I think your issue is that you are returning the response instead of calling
callback(null, response);
where callback should be the third argument to your lambda handler func:
def lambda_handler(event, context, callback):
Since content-length is not mutable, we should assume (and I checked this is true in practice at least for viewer request functions), cloudfront will generate it for you when you generate a response in the edge function.

Curl command for creating Github Repository is not working

I tried the following code for creating a repository. But it gives JSON parsing problem. What causes this?. I took this code from https://docs.github.com/en/rest/reference/repos tried in python code using requests.post method. But that too gave the same error Problem Parsing JSON. Is this bad request or the parsing problem is inside Github API.
curl -i -H "Authorization: token MYACCESSTOKENHERE" \
-d '{ \
"name": "simp", \
"auto_init": true, \
"private": true, \
"gitignore_template": "nanoc" \
}' \
https://api.github.com/user/repos
Output:
HTTP/1.1 400 Bad Request
Date: Sun, 09 Aug 2020 05:14:23 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 159
Server: GitHub.com
Status: 400 Bad Request
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4956
X-RateLimit-Reset: 1596950918
X-OAuth-Scopes: admin:gpg_key, admin:org, admin:org_hook, admin:repo_hook, gist, notifications, read:packages, repo, user, workflow, write:discussion, write:packages
X-Accepted-OAuth-Scopes: public_repo, repo
X-GitHub-Media-Type: github.v3; format=json
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
Vary: Accept-Encoding, Accept, X-Requested-With
X-GitHub-Request-Id: B649:25C9:CDEF3A:117E0EE:5F2F862F
{
"message": "Problems parsing JSON",
"documentation_url": "https://docs.github.com/rest/reference/repos#create-a-repository-for-the-authenticated-user"
}
Could you please validate your token is authorized.
Please refer Authorizing OAuth Apps
curl -H "Authorization: token OAUTH-TOKEN" <https://<your_repo_endpoint> -I
Update using JSON formatter
"-d""{ \\
\"name\": \"simp\", \\
\"auto_init\": true, \\
\"private\": true, \\
\"gitignore_template\": \"nanoc\" \\
}""\\
The problem is with the JSON format. After I removed unwanted lines and spaces from the curl command, it worked.
curl -i -H "Authorization: token 1f003ae8eab2feea72630d6f3150b921a522a868" -d '{ "name": "simp1", "private": true}' https://api.github.com/user/repos

Item Type HTTP Agent not parsing Json Response

I have a C# application that return a Json status when I call "http://host:port/app-status".
The response looks like:
{
"prtg":
{
"result": [
{
"channel": "DDS - ZDM - Konsistenzprüfung",
"value": "3",
"valuelookup": "prtg.RCLookup.DDS_ZDM_Check.BitField"
},
{
"channel": "ZDM DB Verbindungsversuche",
"value": "0",
"valuelookup": "prtg.RCLookup.Default.DB.Connect.Retry"
}
]
}
}
Then i have on zabbix server an item which type is http agent.
The request works fine. But I get this error:
Preprocessing failed for: HTTP/1.1 200 Ok..Content-Length: 361..Content-Type: application/json..Server: Grapevine/4.1.1.0 M...
1. Failed: cannot extract value from json by path ".prtg.result[0].value": cannot parse as a valid JSON object: invalid object format, expected opening character '{' or '[' at: 'HTTP/1.1 200 Ok
Content-Length: 361
Content-Type: application/json
Server: Grapevine/4.1.1.0 Microsoft-HTTPAPI/2.0
Date: Fri, 12 Apr 2019 14:19:12
In the preprocessing tab I have set a processing step with JsonPath.
The JsonPath is: .prtg.result[0].value
What is wrong?
Can help me everybody?
from error message it seems that you are trying to parse response from server inclusive headers which is wrong - you need to parse as JSON only data which is sent back (exclusive headers)

Google Drive Doc Fill/AppScripts

I've been working with the Google Drive API in my Java application. The goal is to have a framework for our internal AppEngine project to fill out various templates in our organization's Team Drives.
I have setup a Service Account with domain-wide permissions and have gone through the "Executing Functions using the Apps Script API" documents. I can use the Java v3 library to update a sheet to get some number information, copy the template to a new file in the Team Drive. I've written a simple AppScript function to take in the Docs File Id and a JSON string representing the token/value pairs to substitute.
Everything goes great until it's time to execute the script when I get:
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Request contains an invalid argument.",
"reason" : "badRequest"
} ],
"message" : "Request contains an invalid argument.",
"status" : "INVALID_ARGUMENT"
}
Not sure how to debug or change this error, as I don't really get any additional info. I do see the request being made in the AppScript API and project console, but don't know how to get more information.
My code to execute is as follows:
List<Object> params = Lists.newArrayList();
params.add( fileId );
params.add( json );
ExecutionRequest request = new ExecutionRequest().setFunction( "main2" )
.setParameters( params );
script().scripts().run( TEMPLATE_FILLER_SCRIPT_ID, request ).execute();
After enabling the logging to I was able to pull the entire web request. It all looks ~fine but maybe there's more here:
CONFIG: {"function":"main2","parameters":["15gXG9frqV0VrF57BjDqOe0pCHPWxVJ1ucV0Fl3zR0J4","{\"num\":\"035\",\"digital_key\":\"https://docs.google.com/document/d/15gXG9frqV0VrF57BjDqOe0pCHPWxVJ1ucV0Fl3zR0J4\",\"Title\":\"Build Control for Image Enhancement\",\"Author\":\"Evan Ruff\",\"Date\":\"05/11/18\"}"]}
May 11, 2018 4:59:41 PM com.google.api.client.http.HttpRequest execute
CONFIG: -------------- REQUEST --------------
POST https://script.googleapis.com/v1/scripts/1T7HAaK2yJ2qUf3sOAz3ZGSPRql73-DGCe79-dLgCyqbnh_LbIn5KgQ4r:run
Accept-Encoding: gzip
Authorization: Bearer *******MY TOKEN *********
User-Agent: Memo Builder/1.0 Google-API-Java-Client Google-HTTP-Java-Client/1.23.0 (gzip)
Content-Type: application/json; charset=UTF-8
Content-Encoding: gzip
Content-Length: 230
May 11, 2018 4:59:41 PM com.google.api.client.http.HttpRequest execute
CONFIG: curl -v --compressed -X POST -H 'Accept-Encoding: gzip' -H 'Authorization: Bearer ya29.c.EmW4BdRkBR2JFBoDaAw_FG8DFbNCHYoe4E3jBs9HyowMAPqM2SnNky4ffRdh0zxG2nc4ylcIlr9yUHJ-ibOJuXdJhakgTmEyC7R4xn8cdKEif7mSeaeRGV9XwYI4W3AkoRAz-sCWmw' -H 'User-Agent: Micro C-Memo Builder/1.0 Google-API-Java-Client Google-HTTP-Java-Client/1.23.0 (gzip)' -H 'Content-Type: application/json; charset=UTF-8' -H 'Content-Encoding: gzip' -d '#-' -- 'https://script.googleapis.com/v1/scripts/1T7HAaK2yJ2qUf3sOAz3ZGSPRql73-DGCe79-dLgCyqbnh_LbIn5KgQ4r:run' << $$$
May 11, 2018 4:59:41 PM com.google.api.client.util.LoggingByteArrayOutputStream close
CONFIG: Total: 299 bytes
May 11, 2018 4:59:41 PM com.google.api.client.util.LoggingByteArrayOutputStream close
CONFIG: {"function":"main2","parameters":["15gXG9frqV0VrF57BjDqOe0pCHPWxVJ1ucV0Fl3zR0J4","{\"num\":\"035\",\"digital_key\":\"https://docs.google.com/document/d/15gXG9frqV0VrF57BjDqOe0pCHPWxVJ1ucV0Fl3zR0J4\",\"Title\":\"Build Control for Image Enhancement\",\"Author\":\"Evan Ruff\",\"Date\":\"05/11/18\"}"]}
May 11, 2018 4:59:41 PM com.google.api.client.http.HttpResponse <init>
CONFIG: -------------- RESPONSE --------------
HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
X-Frame-Options: SAMEORIGIN
Alt-Svc: hq=":443"; ma=2592000; quic=51303433; quic=51303432; quic=51303431; quic=51303339; quic=51303335,quic=":443"; ma=2592000; v="43,42,41,39,35"
Cache-Control: private
Server: ESF
X-Content-Type-Options: nosniff
Content-Encoding: gzip
Vary: Referer
Vary: X-Origin
Vary: Origin
X-XSS-Protection: 1; mode=block
Date: Fri, 11 May 2018 20:59:41 GMT
Content-Type: application/json; charset=UTF-8
This seems simple enough but I'm really struggling. Any tips would be appreciated!
In case someone comes across this, Google released a Java Docs API soon after I actually got this working. The Docs API is a much nicer solution.

DropWizard HealthCheck Json Response

I'm using dropwizard 0.6.2 for my service. The healthcheck response from dropwizard returns plain text. And I found a question in stackoverflow which had an answer that says we can pass a ObjectMapper to a healthcheck. But I couldn't able to find a way to pass the ObjectMapper to the HealthCheck.
Is there a way to return the healthcheck response in JSON?
As of Dropwizard 0.7, the /healthcheck path returns a JSON response:
HTTP/1.1 200 OK
Cache-Control: must-revalidate,no-cache,no-store
Content-Length: 299
Content-Type: application/json
Date: Thu, 14 Aug 2014 07:55:29 GMT
{
"My custom HealthCheck":
{
"healthy": true,
"message": "your message here"
},
"deadlocks":
{
"healthy": true
},
"storage":
{
"healthy": true
}
}
The Dropwizard use the codehale HealthCheck class.
You can call Result.healthy() and passing for parameter your JSON string.
In the method that you call the healthcheck you can use:
Result.healthy("your json");