I have an interesting problem.
I have this document:
{
"correlationId": "6298865a73b477106c98d021",
"leg": 0,
"tag": "sent",
"offset": 322858,
"len": 178,
"prev": {
"page": {
"file": 10352,
"page": 2
},
"record": 911
},
"data": "HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess Denied"
}
But because the "data" element could contain also regular JSON object I would like to move the value
"HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54
GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID:
Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess
Denied"
into "data": { "message": "HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess Denied"} structure.
I tried to find a solution with jq but I didn't found any filter which would move it.
Any idea please?
It seems you want:
.data |= { message: .}
Related
How do I map the following in dataweave 2.0 .. I am able to split the first element .. I need to form my list based of the first element in my input and parse out the remaining under appropriate indexes. for e.g. when there is a 5th value something.pem, there will be subsequent values for it and the output array need to be populated
%dw 2.0
output application/json
---
payload[0] splitBy (/\s/)
map (certs, index) -> {
certsname: certs,
validfrome: (payload filter ($$<=4)) [index+1],
validto: (payload filter ($$<=4)) [index+2],
issuer: (payload filter ($$<=4)) [index+3],
subject: (payload filter ($$<=5)) [index+4]
}
Input
[
"auth0.pem maximo.cer synovos.pem veevavault.pem",
"notBefore=Apr 30 00:00:00 2020 GMT",
"notAfter=May 30 12:00:00 2021 GMT",
"issuer= /C=US/O=Amazon/OU=Server CA 1B/CN=Amazon",
"subject= /CN=auth0.com",
"notBefore=Feb 5 18:02:21 2020 GMT",
"notAfter=Apr 5 18:44:22 2021 GMT",
"issuer= /C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./OU=http://certs.starfieldtech.com/repository//CN=Starfield Secure Certificate Authority - G2",
"subject= /OU=Domain Control Validated/CN=*.corp.amazon.com",
"notBefore=Aug 1 17:57:31 2020 GMT",
"notAfter=Aug 15 17:57:31 2020 GMT",
"issuer= /C=US/ST=California/O=Zscaler Inc./OU=Zscaler Inc./CN=Zscaler Intermediate Root CA (zscloud.net) (t)",
"subject= /OU=Domain Control Validated/CN=*.synovos.com",
"notBefore=Jan 23 00:00:00 2018 GMT",
"notAfter=Jan 27 12:00:00 2021 GMT",
"issuer= /C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA",
"subject= /C=US/ST=California/L=Pleasanton/O=Veeva Systems, Inc./OU=Vault/CN=*.veevavault.com"
]
to an Output
[
{
"certsname": "auth0.pem",
"validfrome": "notBefore=Apr 30 00:00:00 2020 GMT",
"validto": "notAfter=May 30 12:00:00 2021 GMT",
"issuer": "issuer= /C=US/O=Amazon/OU=Server CA 1B/CN=Amazon",
"subject": "subject= /CN=auth0.com"
},
{
"certsname": "maximo.cer",
"validfrom": "notBefore=Feb 5 18:02:21 2020 GMT",
"validto": "Apr 5 18:44:22 2021 GMT",
"issuer": "/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./OU=http://certs.starfieldtech.com/repository//CN=Starfield Secure Certificate Authority - G2\"",
"subject": "subject= /OU=Domain Control Validated/CN=*.corp.amazon.com"
},
{
"certsname": "synovos.pem",
"validfrom": "notBefore=Aug 1 17:57:31 2020 GMT",
"validto": "notAfter=Aug 15 17:57:31 2020 GMT",
"issuer": "issuer= /C=US/ST=California/O=Zscaler Inc./OU=Zscaler Inc./CN=Zscaler Intermediate Root CA (zscloud.net) (t)",
"subject": "subject= /OU=Domain Control Validated/CN=*.synovos.com"
},
{
"certsname": "veevavault.pem",
"validfrom": "notBefore=Jan 23 00:00:00 2018 GMT",
"validto": "notAfter=Jan 27 12:00:00 2021 GMT",
"issuer": "issuer= /C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA",
"subject": "subject= /C=US/ST=California/L=Pleasanton/O=Veeva Systems, Inc./OU=Vault/CN=*.veevavault.com"
}
]
Maybe you can optimize this.
%dw 2.0
output application/json
var validFrom = ((payload map {
(($$): $) if($ contains("notBefore"))
}) - ({}))
var validTo = ((payload map {
(($$): $) if($ contains("notAfter"))
}) - ({}))
var issuer = ((payload map {
(($$): $) if($ contains("issuer"))
}) - ({}))
var subject = ((payload map {
(($$): $) if($ contains("subject"))
}) - ({}))
---
payload[0] splitBy (/\s/) map {
certsname: $,
validfrom: validFrom[$$][0],
validto: validTo[($$)][0],
issuer: issuer[$$][0],
subject: subject[$$][0]
}
The input array is a pretty bad structure. I advice to make it more structured instead. If you can't change it, then below is the script I made to get the expected output.
Script:
%dw 2.0
output application/json
import * from dw::core::Arrays
import * from dw::core::Strings
var certs=payload[0] splitBy (/\s/)
var attributes=payload[1 to sizeOf(payload)-1]
var certAttributes=attributes divideBy 4 // assume there are exactly 4 attributes per certificate
fun splitAttribute(a)={ (substringBefore(a,'=')): substringAfter (a, '=')}
fun attributesToObject(x)=x reduce ((item, accumulator={}) -> accumulator ++ item)
---
certAttributes map (
{certsname: certs[$$]} ++ attributesToObject( $ map splitAttribute($))
)
Output:
[
{
"certsname": "auth0.pem",
"notBefore": "Apr 30 00:00:00 2020 GMT",
"notAfter": "May 30 12:00:00 2021 GMT",
"issuer": " /C=US/O=Amazon/OU=Server CA 1B/CN=Amazon",
"subject": " /CN=auth0.com"
},
{
"certsname": "maximo.cer",
"notBefore": "Feb 5 18:02:21 2020 GMT",
"notAfter": "Apr 5 18:44:22 2021 GMT",
"issuer": " /C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./OU=http://certs.starfieldtech.com/repository//CN=Starfield Secure Certificate Authority - G2",
"subject": " /OU=Domain Control Validated/CN=*.corp.amazon.com"
},
{
"certsname": "synovos.pem",
"notBefore": "Aug 1 17:57:31 2020 GMT",
"notAfter": "Aug 15 17:57:31 2020 GMT",
"issuer": " /C=US/ST=California/O=Zscaler Inc./OU=Zscaler Inc./CN=Zscaler Intermediate Root CA (zscloud.net) (t)",
"subject": " /OU=Domain Control Validated/CN=*.synovos.com"
},
{
"certsname": "veevavault.pem",
"notBefore": "Jan 23 00:00:00 2018 GMT",
"notAfter": "Jan 27 12:00:00 2021 GMT",
"issuer": " /C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA",
"subject": " /C=US/ST=California/L=Pleasanton/O=Veeva Systems, Inc./OU=Vault/CN=*.veevavault.com"
}
]
Try this code:
%dw 2.0
output application/json
fun getValue(key) = payload[?($ contains key)]
---
payload[0] splitBy (" ") map {
certsname: $,
validfrom: getValue("notBefore")[($$)],
validto: getValue("notAfter")[($$)],
issuer: getValue("issuer")[($$)],
subject: getValue("subject")[($$)]
}
I am zero in regex and need help in parsing the value for key "access_token" from the below output.This will be later used in passing as variable for another function.
So basically the regex should only fetch
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJodHRwOi8vc2NoZW1hcy54bWxzb2FwLm9yZy93cy8yMDA1LzA1L2lkZW50aXR5L2NsYWltcy9uYW1lIjoiaXRwcm9kbW9uaXRvciIsImh0dHA6Ly9zY2hlbWFzLnhtbHNvYXAub3JnL3dzLzIwMDUvMDUvaWRlbnRpdHkvY2xhaW1zL2dpdmVubmFtZSI6IkxPS0VTSCBEVVJBSVJBSiIsImh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vd3MvMjAwOC8wNi9pZGVudGl0eS9jbGFpbXMvcm9sZSI6IlNEQ1NfSEVMUERFU0siLCJQZXJtaXNzaW9ucyI6IjkwMzAsOTAwMCw5MDI4LDkwMjcsOTAyNiw5MDI1LDkwMjQsOTAyMyw5MDIyLDkwMjEsOTAyMCw5MDE5LDkwMTgsOTAxMyw5MDEyLDkwMTEsOTAxMCw5MDA5LDkwMDgsOTAwNyw5MDA2LDkwMDUsOTAwNCw5MDAzLDkwMDIsOTAwMSw5MDI5IiwiYWlycG9ydHMiOiJTWVoiLCJjbGllbnRJUCI6IjEwLjExMS4xLjEiLCJlbnYiOiJQUk9EIiwicmVzQ2hhbm5lbElEIjoiMTkiLCJpc0FwdENsbnQiOiJUcnVlIiwiY2hrTGNuIjoiQWlycG9ydCIsIm5iZiI6MTUzODkxNDY0OSwiZXhwIjoxNTM4OTE3NjQ5LCJpc3MiOiJmbHlkdWJhaS5jb20iLCJhdWQiOiIxNDEyMDAxIn0.qYID1b5lMjFhn7fTcSX5v6K6z2YpGJwAvE4gQfVrhxo
Here is the output of my Post output
{
"access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJodHRwOi8vc2NoZW1hcy54bWxzb2FwLm9yZy93cy8yMDA1LzA1L2lkZW50aXR5L2NsYWltcy9uYW1lIjoiaXRwcm9kbW9uaXRvciIsImh0dHA6Ly9zY2hlbWFzLnhtbHNvYXAub3JnL3dzLzIwMDUvMDUvaWRlbnRpdHkvY2xhaW1zL2dpdmVubmFtZSI6IkxPS0VTSCBEVVJBSVJBSiIsImh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vd3MvMjAwOC8wNi9pZGVudGl0eS9jbGFpbXMvcm9sZSI6IlNEQ1NfSEVMUERFU0siLCJQZXJtaXNzaW9ucyI6IjkwMzAsOTAwMCw5MDI4LDkwMjcsOTAyNiw5MDI1LDkwMjQsOTAyMyw5MDIyLDkwMjEsOTAyMCw5MDE5LDkwMTgsOTAxMyw5MDEyLDkwMTEsOTAxMCw5MDA5LDkwMDgsOTAwNyw5MDA2LDkwMDUsOTAwNCw5MDAzLDkwMDIsOTAwMSw5MDI5IiwiYWlycG9ydHMiOiJTWVoiLCJjbGllbnRJUCI6IjEwLjExMS4xLjEiLCJlbnYiOiJQUk9EIiwicmVzQ2hhbm5lbElEIjoiMTkiLCJpc0FwdENsbnQiOiJUcnVlIiwiY2hrTGNuIjoiQWlycG9ydCIsIm5iZiI6MTUzODkxNDY0OSwiZXhwIjoxNTM4OTE3NjQ5LCJpc3MiOiJmbHlkdWJhaS5jb20iLCJhdWQiOiIxNDEyMDAxIn0.qYID1b5lMjFhn7fTcSX5v6K6z2YpGJwAvE4gQfVrhxo",
"token_type": "bearer",
"expires_in": 2999,
"refresh_token": "65be41084c0b4adeaeec9725cb2e6240",
"audience": "1412001",
"displayName": "Lokesh",
"userId": "testuser78",
"rolesandpermission": "HELPDESK:9030,9000,9028,9027,9026,9025,9024,9023,9022,9021,9020,9019,9018,9013,9012,9011,9010,9009,9008,9007,9006,9005,9004,9003,9002,9001,9029",
"resChannelID": "19",
"Client": "FYC",
"isAptClnt": "True",
"scope": "apt:FYC env:PROD role:HELPDESK",
".issued": "Sun, 07 Oct 2018 12:17:29 GMT",
".expires": "Sun, 07 Oct 2018 13:07:29 GMT"
}
Simple PCRE look like
\"access_token\":.\"(.+?)\"
And you will get your token in fast captured group
You can practice with this regex on this website
https://regex101.com/
I would like to add new path into existing document
./jq < test.json
{
"correlationId": "6298865a73b477106c98d021",
"leg": 0,
"tag": "sent",
"offset": 322858,
"len": 178,
"prev": {
"page": {
"file": 10352,
"page": 2
},
"record": 911
},
"data": "HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess Denied"
}
I am using filter setpath described in jq manual.
But even if I copied the documented string
./jq 'setpath([0,"a"]; 1)' < test.json
still getting error:
jq: error (at <stdin>:1): Cannot index object with number
I do not see there any syntax issue. Did I overlooked something?
Regards and thanks, Reddy
You cannot use integer indices as keys in the JSON object. For JSON objects, the key must be a string, so you could write:
jq 'setpath(["0","a"]; 1)' < test.json
Output:
{
"correlationId": "6298865a73b477106c98d021",
"leg": 0,
"tag": "sent",
"offset": 322858,
"len": 178,
"prev": {
"page": {
"file": 10352,
"page": 2
},
"record": 911
},
"data": "HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess Denied",
"0": {
"a": 1
}
}
Good day, does anyone know typical JSON response data, when accessing a file? I am more interested in whether or not one can check if a response object is a file or a directory!
Thanx mates
This is a common JSON answer, when you try to get information about file/folder.
You can see more information about request here
if is_dir is true, it's a folder.
{
"size": "225.4KB",
"rev": "35e97029684fe",
"thumb_exists": false,
"bytes": 230783,
"modified": "Tue, 19 Jul 2011 21:55:38 +0000",
"client_mtime": "Mon, 18 Jul 2011 18:04:35 +0000",
"path": "/Getting_Started.pdf",
"is_dir": false,
"icon": "page_white_acrobat",
"root": "dropbox",
"mime_type": "application/pdf",
"revision": 220823
}
I tried this one which I found here -
http://puppygifs.tumblr.com/api/read/json
But if I test its validity, it gives me an error.
I validated it here:
http://jsonlint.com/
Can anyone point me to a functional JSON link that I can use to test my applications?
Here's a 'My IP' type service that returns JSON.
http://whoami.bradallen.net/
Why not try out the free stackexchange api?
burhan#lenux:~$ http "http://api.stackexchange.com/2.1/users/1755023?site=stackoverflow"
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: false
Access-Control-Allow-Methods: GET, POST
Access-Control-Allow-Origin: *
Cache-Control: private
Content-Encoding: gzip
Content-Length: 433
Content-Type: application/json; charset=utf-8
Date: Thu, 25 Jul 2013 16:54:31 GMT
{
"has_more": false,
"items": [
{
"accept_rate": 91,
"account_id": 1950909,
"age": 27,
"badge_counts": {
"bronze": 9,
"gold": 0,
"silver": 0
},
"creation_date": 1350528803,
"display_name": "Rj.",
"is_employee": false,
"last_access_date": 1374770664,
"last_modified_date": 1372353815,
"link": "http://stackoverflow.com/users/1755023/rj",
"location": "West Hollywood, CA",
"profile_image": "https://www.gravatar.com/avatar/f1f3ab1c31c6ee987ba28a78318586fd?d=identicon&r=PG",
"reputation": 423,
"reputation_change_day": 5,
"reputation_change_month": 95,
"reputation_change_quarter": 95,
"reputation_change_week": 5,
"reputation_change_year": 315,
"user_id": 1755023,
"user_type": "registered",
"website_url": ""
}
],
"quota_max": 300,
"quota_remaining": 294
}