Zapier Catch (Raw) Hook JSON parsing issue - json

I would like to configure sync between two different CRMs (Clevertap and Intercom) using Zapier and Webhooks. In general Clevertap sends the following JSON to webhook:
{
"targetId": 1548328164,
"profiles": [
{
"event_properties": {
"MSG-sms": true,
"MSG-push": true,
"businessRole": "EMPLOYEE",
"Mobile Number": "123123123123",
"Name": "Artem Hovtvianisa",
"Title": "Mr",
"Last Name": "Hovtvianisa",
"Gender": "M",
"Customer type": "Business Account Holder",
"MSG-email": true,
"First Name": "Artem",
"Last seen IP": "111.177.74.50",
"tz": "GMT+0200",
"International customer": "yes",
"isBusiness": true,
"Email": "xxxyyy#gmail.com",
"Identity": 15675
},
"objectId": "e32e4de3c1e84b2d9bab3707c92cd092",
"all_identities": [
"15675",
"xxxyyy#gmail.com"
],
"email": "xxxyyy#gmail.com",
"identity": "15675"
}
]
}
Zapier provides two types of catch webhook: regular and Raw.
Catch Raw Hook
When I use this type, JSON raw data will be handled OK and on the next step (Zapier JS code app) I am able to pass proper JSON data like in example above.
However when I use simple JS code to parse JSON object and get profiles[0] array value I get the following error "TypeError: Cannot read property '0' of undefined"
JS Code from Code step:
var result = JSON.parse(JSON.stringify(inputData));
console.log(result.profiles[0]);
return result;
Catch Hook
In case I use regular Catch Hook, hook parse data in some odd way, like this:
JSON.parse cannot recognize this structure.
Please advise how can I handle Webhook Zapier step in a proper way in order to get profiles[0] array item values?
Thanks in advance!

David here, from the Zapier Platform team. You're on the right track!
Catch Raw Hook is the way to go here. Your issue is that the data is coming in as a string and you're re-stringifying it before parsing it, which gets you back to where you came from. A simpler version:
JSON.stringify("asdf") // => "\"asdf\"", quotes in the string
JSON.parse("\"asdf\"") // => "asdf", the original string
"asdf".profiles // => undefined
undefined[0] // => error, cannot read property "0" of undefined
Instead, just parse it and you're good to go!
// all your variables are already in "inputData", so yours,
// also named inputData, must be referenced explicitly.
const result = JSON.parse(inputData.inputData);
return {result: result, firstProfile: result.profiles[0]};

Related

422 error trying to save json data to the database

I'm trying to save data to my MySql db from a Node method. This includes a field called attachments.
console.log(JSON.stringify(post.acf.attachments[0])); returns:
{
"ID": 4776,
"id": 4776,
"title": "bla",
"filename": "bla.pdf",
"filesize": 1242207,
"url": "https://example.com/wp-content/uploads/bla.pdf",
"link": "https://example.com/bla/",
"alt": "",
"author": "1",
"description": "",
"caption": "",
"name": "bla",
"status": "inherit",
"uploaded_to": 0,
"date": "2020-10-23 18:05:13",
"modified": "2020-10-23 18:05:13",
"menu_order": 0,
"mime_type": "application/pdf",
"type": "application",
"subtype": "pdf",
"icon": "https://example.com/wp-includes/images/media/document.png"
}
This is indeed the data I want to save to the db:
await existing_post.save({
...
attachments: post.acf.attachments[0],
)};
However, the attachments field produces a 422 server error (if I comment out this field, the other fields save without a problem to the db). I'm not getting what is causing this error. Any ideas?
I've also tried
await existing_post.save({
...
attachments: post.acf.attachments,
)};
but then it seems to just save "[object Object]" to the database.
The field in the database is defined as text. I've also tried it by defining the field as json, but that made no difference.
exports.up = function (knex, Promise) {
return knex.schema.table("posts", function (table) {
table.longtext("attachments");
});
};
The 422 error code is about the server unable to process the data you are sending to it. In your case, your table field is longtext when post.acf.attachments seems like an object. That's why it saves [object Object] to your db (It is the return value of the toString() method).
Try using
await existing_post.save({
...
attachments: JSON.stringify(post.acf.attachments),
)};
MySQL and knex both support the JSON format, I'd suggest you change the field to json. (See knex docs and mysql 8 docs). You'll stiil need to stringify your objects tho.
EDIT: I just saw that Knex supports jsonInsert (and plenty other neat stuff) as a query builder that should be useful for you.
Mysql also support a large range of cool stuffs for handling jsons
In addition, when you fetch the results in the database, you'll need to parse the JSON result to get an actual JSON object:
const acf = await knex('posts').select('acf').first();
const attachment = JSON.parse(acf.attachment;
Knex also provide jsonExtract that should fill your needs (See also the mysql json_extract

How to parse nested JSON, within a string, using Kusto

I have a Python Azure Function that produces custom logging messages when the Function executes. I need to pull out some of the JSON values nested in the logging message.
How can I use Kusto to access the nested JSON within the logging message string?
Example logging message:
Desired values marked with <----------
####### EventGrid trigger processing an event:
{
"id": "long-guid",
"data": {
"api": "FlushWithClose",
"requestId": "long-guid",
"eTag": "long-guid",
"contentType": "application/octet-stream",
"contentLength": 16264, <----------------------
"contentOffset": 0,
"blobType": "BlockBlob",
"blobUrl": "https://function.blob.core.windows.net/parentdir/childdir/file.name",
"url": "https://function.dfs.core.windows.net/parentdir/childdir/file.name", <---- JUST FILE.NAME here
"sequencer": "long-guid",
"identity": "long-guid",
"storageDiagnostics": {
"batchId": "long-guid"
}
},
"topic": "/subscriptions/long-guid/resourceGroups/resourceGroup/providers/Microsoft.Storage/storageAccounts/accountName",
"subject": "/blobServices/default/containers/containerName/blobs/childDir/file.name",
"event_type": "Microsoft.Storage.BlobCreated"
} #######
I imagine it has something to do with the Kusto extend function, but piping in...
| extend parsedMessage = todynamic(message)
| project timestamp, test = parsedMessage["id"]
...yields only an empty test column
message in your specific case isn't a valid JSON payload - as it has the ###... EventGrid trigger processing an event: prefix (and a somewhat similar suffix).
That is why todynamic() isn't able to process it and why you're not able to reference properties in the JSON payload that's included in it.
Ideally, you would change the payload you ingest to be a valid JSON payload, and re-type the target column to dynamic instead of string.
If you can't do that, you can use the substring() function or parse operator to get everything but the aforementioned prefix/suffix, and parse the output of that using todynamic()
though note that doing that each time you query the data bears runtime overhead that could be avoided by following the advice above.

Cypress intercept API JSON response and extract URL

My web app sends an API POST request to create an application and returns JSON response. I want to access one particular JSON object from that response.
My JSON starts like this
[
{
"status_code": 201,
"body": {
"created": "2021-01-28T00:00:00Z",
"modified": "2021-01-28T00:00:00Z",
"id": "a2d86d17-9b3c-4c4d-ac49-5b9d8f6d6f8f",
"applicant": {
"id": "07f1e1d3-0521-401b-813e-3f777f2673c6",
"status": "Pending",
"first_name": "",
"last_name": "",
"URL": "some onboarding url"
And I wanna take that URL in the JSON response and visit it later in my cypress automation script.
Notice that the JSON repsonse starts with a square bracket not a curly bracket, which means, the whole response is an object, I assume?
My cypress script looks like this
cy.contains('button', 'CONTINUE').click()
cy.EmailGen('candidate').then(email => {
cy.get('#emails\\[0\\]').type(`${email}`)
cy.wrap(email).as('candidateEmail')
})
//writing intercept here, before the Send application button, which triggers the POST req.
cy.intercept('/hr/v1/applications/invite').as('getURL')
cy.get('button').contains('SEND APPLICATION').click({ force: true })
//waiting on the alias of intercept and using interception to print objects from response
cy.wait('#getURL').then((interception)=> {
cy.log(interception.response.body.applicant.URL)
})
cy.Logout()
The script executes with no errors. Just that nothing is logged in the cy.log statement. Below is the screen.
I also tried using another method as given below.
cy.intercept('/hr/v1/applications/invite',(req) => {
req.reply((res=> {
expect(res.status_code).to.equal('201')
expect(res.body.applicant.status).to.equal('Pending')
}))
})
In this case, I get a assert error embedded with the request and response along with some other stuff which I am unable to understand.
The complete error goes something like this...
"expected The following error originated from your test code, not from Cypress.\n\n > A response callback passed to req.reply() threw an error while intercepting a response:\n\nexpected undefined to equal '201'\n\nRoute: {\n "matchUrlAgainstPath": true,\n "url": "/hr/v1/applications/invite"\n}\n\nIntercepted request:{} Intercepted response: {} When Cypress detects uncaught errors originating from your test code it will automatically fail the current test. to include window.zE is not a function"
Its a bit weird to read this..
My application sometimes throws this exception, which I have handled using following code.
cy.on('uncaught:exception', (err, runnable) => {
expect(err.message).to.include('window.zE is not a function')
done()
return false
})
I really hope I have explained everything here. Please, help a noob.
Regards
As Richard Matsen suggested in the comments,
I used console.log(interception.response) and checked the console output in the browser controlled by Cypress.
I Noticed that the response json structure was something different than what I got in the network tab of developers tools, while using the web app.
The response was something like below...
{headers: {…}, url: "https://example.com/hr/v1/applications/invite/batch/", method: null, httpVersion: "1.1", statusCode: 200, …}
body: Array(1)
0:
body:
applicant: {id: "c6b2d686-d4f3-483e-abc8-e4641c365845", status: "Pending", first_name: "", last_name: "", email: "qa2+candidate879#example.com", …}
applicant_status: "NONE"
applicant_status_label: "None"
created: "2021-01-29T00:00:00Z"
get_applicant_status_display: "None"
id: "ad2939f5-c8ab-490a-a9e1-b0474de69e2c"
URL: "some url"
This made me modify the json traverse to
interception.response.body[0].body.applicant.URL
If others have a neat way to handle this, please let me know!

Zapier identifying JSON data as string

My question is how can I pull the values for events.payload.media.name?
I am posting to a raw zapier webhook from another app. If I check it using requestb.in it comes through as "Content-Type: application/json". The output is also validating as JSON.
{
"hook":{
"uuid":"1asdfasd5-asdf-4f52-bd31-c7a544897808"
},
"events":[
{
"uuid":"0asdfasdfasdf0",
"type":"viewing_session.turnstile.converted",
"payload":{
"visitor":{
"id":"28b606b_7853753-3868-4f07-9543-70da084452cc-7442322af-407bdc31d8fc-2739"
},
"viewing_session":{
"id":"154284_b40c5358-1faf-40e9-a44e-60aa641a11cd-fd3c69d8d-302471c603f4-8245"
},
"name":null,
"media":{
"url":"https://things.wistia.com/medias/asdfasdf",
"thumbnail":{
"url":"http://embed.wistia.com/deliveries/asdfasdfasdfasdfasdfasdfasd.jpg?image_crop_resized=200x120"
},
"name":"this is what I want!",
"id":"asdfasdfasdf",
"duration":52.872
},
"last_name":null,
"foreign_data":{
},
"first_name":null,
"email":"email#email.com"
},
"metadata":{
"account_id":"asdfasdfasdf"
},
"generated_at":"2017-05-02T07:31:08Z"
}
]
}
However, when I check the typeof data in the output it is telling me that it is a string (see my code to check below). This prevents me from pulling the info out of it using:
return {stuff: typeof inputData.thing.events.payload.media.name};
I'm a huge noob, am I missing something fundamental here?
screenshot to check typeof data
events is an array, so you would access it like this:
inputData.thing.events[0].payload.media.name
is there a way to have the whole payload without creating a new App in Zapier? inputData didn't work

JSON API result format

Folks,
Designing my first API in Node.JS using restify.js. My background is not webapis, pardon my amateur questions. In any case, I would like to have the res.send(data); responses to comply with the http://jsonapi.org/format/ so that my mobile application can start utilizing the api calls. At the moment if you were to call my api, it would return data in the following format:
{"Count":1,"Items":[{"dbsource":{"S":"foo"},"id":{"S":"5002820"},"name":{"S":"fnameblah,lnameblah"},"expiration":{"S":"06/13/2015"},"type":{"S":"bar"}}]}
Actually what you see above is just a return of a DynamoDB Query call.
So the question is... do you use a special library that you can pass data to, which would format and return the data in JSON format. Which in turn you can return it via res.send(data) to the clients, or is it up to us to make 'data' JSON compliant, then return it? At the end of the day we all want the results to look like:
{
"posts": [{
"id": "1",
"title": "Rails is Omakase",
"links": {
"author": "9",
"comments": [ "5", "12", "17", "20" ]
}
}]
}
Thanks!
In server side, stringify JSON object,
//...
res.statusCode = 200;
res.setHeader('Content-Type', 'application/json');
res.send(JSON.stringify(data)); //data is JSON object
res.end();
In client side, parse JSON string accordingly.
EDIT: Corrected response content type.
JSON data from server should be a JSON string
You have to parse it back the JSON format in client.
JSON.parse(string); // return JSON object