How do I map the following in dataweave 2.0 .. I am able to split the first element .. I need to form my list based of the first element in my input and parse out the remaining under appropriate indexes. for e.g. when there is a 5th value something.pem, there will be subsequent values for it and the output array need to be populated
%dw 2.0
output application/json
---
payload[0] splitBy (/\s/)
map (certs, index) -> {
certsname: certs,
validfrome: (payload filter ($$<=4)) [index+1],
validto: (payload filter ($$<=4)) [index+2],
issuer: (payload filter ($$<=4)) [index+3],
subject: (payload filter ($$<=5)) [index+4]
}
Input
[
"auth0.pem maximo.cer synovos.pem veevavault.pem",
"notBefore=Apr 30 00:00:00 2020 GMT",
"notAfter=May 30 12:00:00 2021 GMT",
"issuer= /C=US/O=Amazon/OU=Server CA 1B/CN=Amazon",
"subject= /CN=auth0.com",
"notBefore=Feb 5 18:02:21 2020 GMT",
"notAfter=Apr 5 18:44:22 2021 GMT",
"issuer= /C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./OU=http://certs.starfieldtech.com/repository//CN=Starfield Secure Certificate Authority - G2",
"subject= /OU=Domain Control Validated/CN=*.corp.amazon.com",
"notBefore=Aug 1 17:57:31 2020 GMT",
"notAfter=Aug 15 17:57:31 2020 GMT",
"issuer= /C=US/ST=California/O=Zscaler Inc./OU=Zscaler Inc./CN=Zscaler Intermediate Root CA (zscloud.net) (t)",
"subject= /OU=Domain Control Validated/CN=*.synovos.com",
"notBefore=Jan 23 00:00:00 2018 GMT",
"notAfter=Jan 27 12:00:00 2021 GMT",
"issuer= /C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA",
"subject= /C=US/ST=California/L=Pleasanton/O=Veeva Systems, Inc./OU=Vault/CN=*.veevavault.com"
]
to an Output
[
{
"certsname": "auth0.pem",
"validfrome": "notBefore=Apr 30 00:00:00 2020 GMT",
"validto": "notAfter=May 30 12:00:00 2021 GMT",
"issuer": "issuer= /C=US/O=Amazon/OU=Server CA 1B/CN=Amazon",
"subject": "subject= /CN=auth0.com"
},
{
"certsname": "maximo.cer",
"validfrom": "notBefore=Feb 5 18:02:21 2020 GMT",
"validto": "Apr 5 18:44:22 2021 GMT",
"issuer": "/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./OU=http://certs.starfieldtech.com/repository//CN=Starfield Secure Certificate Authority - G2\"",
"subject": "subject= /OU=Domain Control Validated/CN=*.corp.amazon.com"
},
{
"certsname": "synovos.pem",
"validfrom": "notBefore=Aug 1 17:57:31 2020 GMT",
"validto": "notAfter=Aug 15 17:57:31 2020 GMT",
"issuer": "issuer= /C=US/ST=California/O=Zscaler Inc./OU=Zscaler Inc./CN=Zscaler Intermediate Root CA (zscloud.net) (t)",
"subject": "subject= /OU=Domain Control Validated/CN=*.synovos.com"
},
{
"certsname": "veevavault.pem",
"validfrom": "notBefore=Jan 23 00:00:00 2018 GMT",
"validto": "notAfter=Jan 27 12:00:00 2021 GMT",
"issuer": "issuer= /C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA",
"subject": "subject= /C=US/ST=California/L=Pleasanton/O=Veeva Systems, Inc./OU=Vault/CN=*.veevavault.com"
}
]
Maybe you can optimize this.
%dw 2.0
output application/json
var validFrom = ((payload map {
(($$): $) if($ contains("notBefore"))
}) - ({}))
var validTo = ((payload map {
(($$): $) if($ contains("notAfter"))
}) - ({}))
var issuer = ((payload map {
(($$): $) if($ contains("issuer"))
}) - ({}))
var subject = ((payload map {
(($$): $) if($ contains("subject"))
}) - ({}))
---
payload[0] splitBy (/\s/) map {
certsname: $,
validfrom: validFrom[$$][0],
validto: validTo[($$)][0],
issuer: issuer[$$][0],
subject: subject[$$][0]
}
The input array is a pretty bad structure. I advice to make it more structured instead. If you can't change it, then below is the script I made to get the expected output.
Script:
%dw 2.0
output application/json
import * from dw::core::Arrays
import * from dw::core::Strings
var certs=payload[0] splitBy (/\s/)
var attributes=payload[1 to sizeOf(payload)-1]
var certAttributes=attributes divideBy 4 // assume there are exactly 4 attributes per certificate
fun splitAttribute(a)={ (substringBefore(a,'=')): substringAfter (a, '=')}
fun attributesToObject(x)=x reduce ((item, accumulator={}) -> accumulator ++ item)
---
certAttributes map (
{certsname: certs[$$]} ++ attributesToObject( $ map splitAttribute($))
)
Output:
[
{
"certsname": "auth0.pem",
"notBefore": "Apr 30 00:00:00 2020 GMT",
"notAfter": "May 30 12:00:00 2021 GMT",
"issuer": " /C=US/O=Amazon/OU=Server CA 1B/CN=Amazon",
"subject": " /CN=auth0.com"
},
{
"certsname": "maximo.cer",
"notBefore": "Feb 5 18:02:21 2020 GMT",
"notAfter": "Apr 5 18:44:22 2021 GMT",
"issuer": " /C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./OU=http://certs.starfieldtech.com/repository//CN=Starfield Secure Certificate Authority - G2",
"subject": " /OU=Domain Control Validated/CN=*.corp.amazon.com"
},
{
"certsname": "synovos.pem",
"notBefore": "Aug 1 17:57:31 2020 GMT",
"notAfter": "Aug 15 17:57:31 2020 GMT",
"issuer": " /C=US/ST=California/O=Zscaler Inc./OU=Zscaler Inc./CN=Zscaler Intermediate Root CA (zscloud.net) (t)",
"subject": " /OU=Domain Control Validated/CN=*.synovos.com"
},
{
"certsname": "veevavault.pem",
"notBefore": "Jan 23 00:00:00 2018 GMT",
"notAfter": "Jan 27 12:00:00 2021 GMT",
"issuer": " /C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA",
"subject": " /C=US/ST=California/L=Pleasanton/O=Veeva Systems, Inc./OU=Vault/CN=*.veevavault.com"
}
]
Try this code:
%dw 2.0
output application/json
fun getValue(key) = payload[?($ contains key)]
---
payload[0] splitBy (" ") map {
certsname: $,
validfrom: getValue("notBefore")[($$)],
validto: getValue("notAfter")[($$)],
issuer: getValue("issuer")[($$)],
subject: getValue("subject")[($$)]
}
Related
I am zero in regex and need help in parsing the value for key "access_token" from the below output.This will be later used in passing as variable for another function.
So basically the regex should only fetch
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJodHRwOi8vc2NoZW1hcy54bWxzb2FwLm9yZy93cy8yMDA1LzA1L2lkZW50aXR5L2NsYWltcy9uYW1lIjoiaXRwcm9kbW9uaXRvciIsImh0dHA6Ly9zY2hlbWFzLnhtbHNvYXAub3JnL3dzLzIwMDUvMDUvaWRlbnRpdHkvY2xhaW1zL2dpdmVubmFtZSI6IkxPS0VTSCBEVVJBSVJBSiIsImh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vd3MvMjAwOC8wNi9pZGVudGl0eS9jbGFpbXMvcm9sZSI6IlNEQ1NfSEVMUERFU0siLCJQZXJtaXNzaW9ucyI6IjkwMzAsOTAwMCw5MDI4LDkwMjcsOTAyNiw5MDI1LDkwMjQsOTAyMyw5MDIyLDkwMjEsOTAyMCw5MDE5LDkwMTgsOTAxMyw5MDEyLDkwMTEsOTAxMCw5MDA5LDkwMDgsOTAwNyw5MDA2LDkwMDUsOTAwNCw5MDAzLDkwMDIsOTAwMSw5MDI5IiwiYWlycG9ydHMiOiJTWVoiLCJjbGllbnRJUCI6IjEwLjExMS4xLjEiLCJlbnYiOiJQUk9EIiwicmVzQ2hhbm5lbElEIjoiMTkiLCJpc0FwdENsbnQiOiJUcnVlIiwiY2hrTGNuIjoiQWlycG9ydCIsIm5iZiI6MTUzODkxNDY0OSwiZXhwIjoxNTM4OTE3NjQ5LCJpc3MiOiJmbHlkdWJhaS5jb20iLCJhdWQiOiIxNDEyMDAxIn0.qYID1b5lMjFhn7fTcSX5v6K6z2YpGJwAvE4gQfVrhxo
Here is the output of my Post output
{
"access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJodHRwOi8vc2NoZW1hcy54bWxzb2FwLm9yZy93cy8yMDA1LzA1L2lkZW50aXR5L2NsYWltcy9uYW1lIjoiaXRwcm9kbW9uaXRvciIsImh0dHA6Ly9zY2hlbWFzLnhtbHNvYXAub3JnL3dzLzIwMDUvMDUvaWRlbnRpdHkvY2xhaW1zL2dpdmVubmFtZSI6IkxPS0VTSCBEVVJBSVJBSiIsImh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vd3MvMjAwOC8wNi9pZGVudGl0eS9jbGFpbXMvcm9sZSI6IlNEQ1NfSEVMUERFU0siLCJQZXJtaXNzaW9ucyI6IjkwMzAsOTAwMCw5MDI4LDkwMjcsOTAyNiw5MDI1LDkwMjQsOTAyMyw5MDIyLDkwMjEsOTAyMCw5MDE5LDkwMTgsOTAxMyw5MDEyLDkwMTEsOTAxMCw5MDA5LDkwMDgsOTAwNyw5MDA2LDkwMDUsOTAwNCw5MDAzLDkwMDIsOTAwMSw5MDI5IiwiYWlycG9ydHMiOiJTWVoiLCJjbGllbnRJUCI6IjEwLjExMS4xLjEiLCJlbnYiOiJQUk9EIiwicmVzQ2hhbm5lbElEIjoiMTkiLCJpc0FwdENsbnQiOiJUcnVlIiwiY2hrTGNuIjoiQWlycG9ydCIsIm5iZiI6MTUzODkxNDY0OSwiZXhwIjoxNTM4OTE3NjQ5LCJpc3MiOiJmbHlkdWJhaS5jb20iLCJhdWQiOiIxNDEyMDAxIn0.qYID1b5lMjFhn7fTcSX5v6K6z2YpGJwAvE4gQfVrhxo",
"token_type": "bearer",
"expires_in": 2999,
"refresh_token": "65be41084c0b4adeaeec9725cb2e6240",
"audience": "1412001",
"displayName": "Lokesh",
"userId": "testuser78",
"rolesandpermission": "HELPDESK:9030,9000,9028,9027,9026,9025,9024,9023,9022,9021,9020,9019,9018,9013,9012,9011,9010,9009,9008,9007,9006,9005,9004,9003,9002,9001,9029",
"resChannelID": "19",
"Client": "FYC",
"isAptClnt": "True",
"scope": "apt:FYC env:PROD role:HELPDESK",
".issued": "Sun, 07 Oct 2018 12:17:29 GMT",
".expires": "Sun, 07 Oct 2018 13:07:29 GMT"
}
Simple PCRE look like
\"access_token\":.\"(.+?)\"
And you will get your token in fast captured group
You can practice with this regex on this website
https://regex101.com/
I have an interesting problem.
I have this document:
{
"correlationId": "6298865a73b477106c98d021",
"leg": 0,
"tag": "sent",
"offset": 322858,
"len": 178,
"prev": {
"page": {
"file": 10352,
"page": 2
},
"record": 911
},
"data": "HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess Denied"
}
But because the "data" element could contain also regular JSON object I would like to move the value
"HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54
GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID:
Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess
Denied"
into "data": { "message": "HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess Denied"} structure.
I tried to find a solution with jq but I didn't found any filter which would move it.
Any idea please?
It seems you want:
.data |= { message: .}
A colleague of mine sent this file over to me asking me to extract the data into a database.... issue is, it is not formatted correctly. (It is supposed to be a json file... even sent as orders.json).
Here is the file contents (The other files are much larger, this is just a snippet):
[
"Buyer: Jane Doe ",
[
"jane Doe",
"Street Here",
"State, ZIP",
"Phone : 8888888"
],
"Ship By: Wed, Oct 11, 2017 to Thu, Oct 12, 2017",
"Deliver By: Mon, Oct 16, 2017 to Thu, Oct 19, 2017",
"Shipping Service: Standard",
[
[
"Product name",
"SKU: 99999999999",
"ASIN: 999999999",
"Condition: New",
"Listing ID: 99999999999",
"Order Item ID: 9999999999",
"Customizations:",
"Size: 16 Inches",
"Monogram Letters: BSA",
"Color: Unpainted (goes out in 48 hrs)"
]
],
"A",
"Wed, Oct 4, 2017, 8:03 PM PT",
"114-7275553-4341048",
[
"1"
]
]
I believe I will have to go through this slowly and use unnecessary methods to get this to look nice and work... unless I am missing something?
As of right now I can't really access the data in a efficient way.
I am building an html application using AngularJS wherein I want to display the json file data on html page. Can anyone help me to Convert GMT date format from json file to hour:minutes AM/PM
Json file looks like this:
[
{
"startTime": "Sun May 24 2015 01:00:00 GMT+0530 (IST)",
"endTime": "Sun May 24 2015 01:30:00 GMT+0530 (IST)",
"title": "Event 1"
},
{
"startTime": "Sun May 24 2015 04:00:00 GMT+0530 (IST)",
"endTime": "Sun May 24 2015 06:00:00 GMT+0530 (IST)",
"title": "Event 2"
},
{
"startTime": "Sat May 23 2015 20:00:00 GMT+0530 (IST)",
"endTime": "Sat May 23 2015 21:30:00 GMT+0530 (IST)",
"title": "Event 3"
},
{
"startTime": "Sat May 23 2015 21:30:00 GMT+0530 (IST)",
"endTime": "Sat May 23 2015 22:15:00 GMT+0530 (IST)",
"title": "Event 4"
},
{
"startTime": "Sun May 24 2015 02:00:00 GMT+0530 (IST)",
"endTime": "Sun May 24 2015 03:00:00 GMT+0530 (IST)",
"title": "Event 5"
}
]
assign your json data to a scope variable:
scope.data = [
{
"startTime": "Sun May 24 2015 01:00:00 GMT+0530 (IST)",
"endTime": "Sun May 24 2015 01:30:00 GMT+0530 (IST)",
"title": "Event 1"
},
...
];
in your html, you can use angularjs date filter like this:
<div>{{ data[0] | date: 'h:m a' }}</div>
The link provided by #worldask says
Date to format either as Date object, milliseconds (string or number) or various ISO 8601
datetime string formats (e.g. yyyy-MM-ddTHH:mm:ss.sssZ and its shorter
versions like yyyy-MM-ddTHH:mmZ, yyyy-MM-dd or yyyyMMddTHHmmssZ). If
no timezone is specified in the string input, the time is considered
to be in the local timezone.
So I am using a custom filter for converting Sun May 24 2015 01:00:00 GMT+0530 (IST) to milliseconds and then using angular filter to show as hh:mm AM/PM
HTML
<div ng-repeat="date in dateJson">
{{date.startTime | myDateFilter | date:'hh: mm a' }}
</div>
dateJson is your sample json...
custom filter
app.filter('myDateFilter', function() {
return function(dateString) {
var date = new Date(dateString)
// converting to milliseconds
dateString = date.getTime();
return dateString;
}
});
SAMPLE PLUNKER
I am trying to load my data from a JSON file into MATLAB that is delimited with ,.
The format of my data is as follows:
{"created_at": "Mon Oct 27 20:35:47 +0000 2014", "tweet": "Silver Finished Up, Gold, Copper, Crude Oil, Nat Gas Down - Live Trading News http://t.co/jNLTUIgHwA", "id": 526834668759285761, "sentiment": "negative"}
{"created_at": "Mon Oct 27 20:36:21 +0000 2014", "tweet": "Gold, Silver slips on lacklustre demand- The Economic Times http://t.co/Jd5Tn9ctfX", "id": 526834810300289024, "sentiment": "negative"}
How would I do so?
As of version 2016b, Matlab has integrated json support.
See:
https://www.mathworks.com/help/matlab/ref/jsondecode.html
In short, you do:
jsonData = jsondecode(fileread('file.json'));
Use jsonLab
One line to read:
tweet_info = loadjson('~/Desktop/test.json')
Here's what's stored in tweet_info{1}
created_at: 'Mon Oct 27 20:35:47 +0000 2014'
tweet: 'Silver Finished Up, Gold, Copper, Crude Oil, Nat Gas Down - Live Trading News http://t.co/jNLTUIgHwA'
id: 5.2683e+17
sentiment: 'negative'
Here's what stored in the test.json file
{"created_at": "Mon Oct 27 20:35:47 +0000 2014", "tweet": "Silver Finished Up, Gold, Copper, Crude Oil, Nat Gas Down - Live Trading News http://t.co/jNLTUIgHwA", "id": 526834668759285761, "sentiment": "negative"}
{"created_at": "Mon Oct 27 20:36:21 +0000 2014", "tweet": "Gold, Silver slips on lacklustre demand- The Economic Times http://t.co/Jd5Tn9ctfX", "id": 526834810300289024, "sentiment": "negative"}