How to pass attributes from Cloud Scheduler to Pub/Sub? - google-cloud-functions

I am trying to trigger a Cloud Function which runs on a schedule as such:
Cloud Scheduler -> Cloud Pub/Sub -> Cloud Functions
My Cloud Function uses attributes from the Pub/Sub message:
let messageTitle = null;
try {
messageTitle = message.attributes.messageTitle;
} catch (e) {
console.error('no title in pub/sub message', e);
}
How can I create a scheduler which has attributes?
I have tried entering this in the Scheduler 'Payload' field:
{
"data": "string",
"attributes": {
messageTitle: "TEST 3 title",
messageBody: "TEST 3 body"
},
"messageId": "string",
"publishTime": "string"
}
But it does not create the attributes in Pub/Sub.

Today (july 2020), an important thing to know on Cloud Scheduler: Few parameters are accessible through the console.
If you want to access to advance parameter, use the gcloud command or the rest API
Now you can access to the retry policies and to the attributes on PubSub message.
gcloud scheduler jobs create pubsub --message-body=my_body \
--attributes=att1=val1,att2=val2 --topic=topicName \
--schedule="0 0 * * *" --time-zone=UTC mySchedulerName
Note, in the console, you can only define the payload, I mean the data field in the PubSub message, equal to the --message-body= in the command line

Related

403 error when attempting to fetch google analytics 4 data from app script

I am attempting to connect a custom community connector to google analytics 4 in order to get data from analytics and be able to modify it in app script and then send it to data studio. However I am having difficulty connecting and retrieving data from google analytics.
The error:
{ error:
{ code: 403,
message: 'Google Analytics Data API has not been used in project 640397821842 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/analyticsdata.googleapis.com/overview?project=640397821842 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.',
status: 'PERMISSION_DENIED',
details: [ [Object], [Object] ] } }
I have the scopes set up
"oauthScopes": [
"https://www.googleapis.com/auth/script.external_request",
"https://www.googleapis.com/auth/analytics.readonly"
],
The code i am trying to test:
function testFetch(){
const options = {
entry: { propertyId: 263290444},
"dateRanges": [{ "startDate": "2020-12-01", "endDate": "2021-03-01" }],
"dimensions": [{ "name": "country" }],
"metrics": [{ "name": "activeUsers" }],
// etc.
}
var response = UrlFetchApp.fetch(
'https://analyticsdata.googleapis.com/v1alpha:runReport', {
method: 'POST',
muteHttpExceptions: true,
headers: {
Authorization : `Bearer ${ScriptApp.getOAuthToken()}`
},
contentType: 'application/json; charset=utf-8',
payload: JSON.stringify(options)
});
var result = JSON.parse(response.getContentText());
console.log(result);
'Google Analytics Data API has not been used in project 640397821842 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/analyticsdata.googleapis.com/overview?project=640397821842 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.',
status:
The problem you are having is that you have not enabled the API in Google developer console You need to follow the link in the error message then go to library and look for Google analytics data api and enable it.
Once you have enabled it try and run your application again.
For those with the same issue who may have overlooked some steps:
Create an API project at https://console.cloud.google.com/
Make sure to connect your appscript project to your cloud API project by changing the Google Cloud Platform (GCP) Project in appscript settings (It is set to a default project when you start).
Enable the required APIs from the API Library in the cloud project.
Now you will be able to fetchurls related to the APIs you have enabled.

Where to view watson assistant error logs

A webhook call fails, how do I see the detailed logs?
The testing webhooks page describes that the errors are put in output.webhook_error.<result_variable>. For example if your result variable is webhook_result_1 then the output is in output.webhook_error.webhook_result_1.
As described in the documentation, if an error happens then in the "Assistant responds" section of the dialog node editor, the "If assistant recognizes" anything_else should be used, so you can put in there The callout generated this error: <? output.webhook_error.webhook_result_1 ?>..
This will show more information. For example if I configure an API to throw an Error, I will get back:
The callout generated this error: {"response_code":400,"message":"Webhook call was not successful. Response code is [400].","response_body":{"code":"cff11d18150e8018d5e304ac3fc35c94","error":"There was an error processing your request."},"content_type":"application/json"}.
If this error happens when the Bot is being used live, then in the logs the webhook response body will be captured. If you use the Try It Out panel in the UI then the logs are not captured.
In the Assistant UI for the Skill in the Analytics tab the conversation is shown. In the "User conversations" part of the UI the text of the conversation is captured, but you will only see the webhooks error if you add output as described above.
The API can be used to fetch logs for a workspace or skill which include the webhook error details. See List Log Events in a Workspace. If you are using a Skill then you can find out the URL to use from the Assistant UI. On the Skills list page, on the particular skill click the ... menu and select View API Details. The Legacy v1 workspace URL: shows the workspace ID to use.
For example if the Legacy v1 workspace URL: is https://gateway.watsonplatform.net/assistant/api/v1/workspaces/ce13f844-c3a7-4f36-97c6-a0ac704024a6/message the URL to use to fetch logs is https://gateway.watsonplatform.net/assistant/api/v1/workspaces/ce13f844-c3a7-4f36-97c6-a0ac704024a6/logs?version=2020-04-01
In the workspace log there will be the details of the user conversation where a webhook error happened in webhook_error:
"output": {
"generic": [
{
"response_type": "text",
"text": ""
},
{
"response_type": "text",
"text": "The callout generated this error: {\"response_code\":400,\"message\":\"Webhook call was not successful. Response code is [400].\",\"response_body\":{\"code\":\"cff11d18150e8018d5e304ac3fc35c94\",\"error\":\"There was an error processing your request.\"},\"content_type\":\"application/json\"}."
},
{
"response_type": "text",
"text": "Customer email is <? context.webhook_result_1.message ?>."
}
],
"text": [
"",
"The callout generated this error: {\"response_code\":400,\"message\":\"Webhook call was not successful. Response code is [400].\",\"response_body\":{\"code\":\"cff11d18150e8018d5e304ac3fc35c94\",\"error\":\"There was an error processing your request.\"},\"content_type\":\"application/json\"}.",
"Customer email is <? context.webhook_result_1.message ?>."
],
"nodes_visited": [
"slot_2_1520179906877",
"handler_3_1520179906877",
"node_1_1520179877410",
"node_9_1530805081298",
"response_5_1604940050659",
"node_7_1520180645829"
],
"webhook_error": {
"webhook_result_1": {
"response_code": 400,
"message": "Webhook call was not successful. Response code is [400].",
"response_body": {
"code": "cff11d18150e8018d5e304ac3fc35c94",
"error": "There was an error processing your request."
},
"content_type": "application/json"
}
},
"nodes_visited_details": [

How to determine why an Azure Function App is not triggered by a webhook

I have:
An JavaScript Azure Function in an HTTP webhook configuration; the Function provides a URL; the Function performs an action
A webhook configured in the software I hope to receive notifications from
An Azure Logic App with an HTTP/webhook step that provides a URL for the webhook notification to go to
My goal is that the Azure Function's URL receives notifications from the software's webhook and performs an action. The Azure Logic App is for testing only.
What works
When the the Azure Logic App's URL is used in the software's webhook configuration, the desired action is performed. All works as expected.
The Azure Logic App's logging shows the JSON output from the incoming webhook. I expect (but believe this may be where I am going wrong) that this is the JSON the webhook is sending to the Azure Logic App's URL. When this JSON is used in the Azure Function UI's "Test" tab > "Request body" field, the desired action is performed. All works as expected.
When the Azure Function's URL and the JSON is in a Postman request, the desired action is performed. All works as expected.
What doesn't work
When the Azure Function's URL is used in the software's webhook configuration, no action is performed. This is of course my goal. From everything I have read, I understand that this URL as a webhook endpoint should work.
Azure Function's URL
This is from Get function URL > default (Function key).
https://<app_name>.azurewebsites.net/api/content?code=<api_key>
Other Azure Function config settings
Allowed HTTP methods: GET, POST
Authorization level: Function
The JSON I believe to be coming over the webhook
{
"headers": {
"Expect": "100-continue",
"Host": "redacted",
"X-Telligent-Webhook-Sender": "redacted",
"Content-Length": "16908",
"Content-Type": "application/json; charset=utf-8"
},
"body": {
"events": [{
"TypeId": "ec9da4f4-0703-4029-b01e-7ca9c9ed6c85",
"DateOccurred": "2018-12-17T22:55:37.7846546Z",
"EventData": {
"ActorUserId": 9999,
"ContentId": "redacted",
"ContentTypeId": "redacted",
"ForumReplyId": 9999,
"ForumThreadId": 9999,
"ForumId": 9999
}
}]
}
}
I also tried with the following test code for the same results. It aligns more closely with the sample payload data provided by the software company:
What I tried
{
"events": [{
"TypeId": "ec9da4f4-0703-4029-b01e-7ca9c9ed6c85",
"DateOccurred": "2018-12-17T22:55:37.7846546Z",
"EventData": {
"ActorUserId": 9999,
"ContentId": "redacted",
"ContentTypeId": "redacted",
"ForumReplyId": 9999,
"ForumThreadId": 9999,
"ForumId": 9999
}
}]
}
Sample payload data
{
"events": [
{
"TypeId": "407ad3bc-8269-493e-ac56-9127656527df",
"DateOccurred": "2015-12-04T16:31:55.5383926Z",
"EventData": {
"ActorUserId": 2100,
"ContentId": "4c792b81-6f09-4a45-be8c-476198ba47be"
}
},
{
"TypeId": "3b75c5b9-4705-4a97-93f5-a4941dc69bc9",
"DateOccurred": "2015-12-04T16:48:03.7343926Z",
"EventData": {
"ActorUserId": 2100,
"ContentId": "4c792b81-6f09-4a45-be8c-476198ba47be"
}
}
]
}
I do not know how to determine why the Azure Function is not triggered by the webhook. The software's API documentation does not seem to provide a way to look at the JSON being sent over the webhook, although in my inexperience I may be wrong.
Is there a mechanism within Azure, or Postman, or another tool that lets me see what JSON is being sent over the webhook? Or perhaps is there another approach to determining the cause of the issue?
Thank you for any help.
This is how I got the JSON file from Azure alerts.
Install Ruby on the server
Install Sinatra with following command gem install sinatra
Create file webhook.rb and paste code bellow
require 'sinatra'
set :port, 80
set :bind, '0.0.0.0'
post '/event' do
status 204 #successful request with no body content
request.body.rewind
request_payload = JSON.parse(request.body.read)
#append the payload to a file
File.open("events.txt", "a") do |f|
f.puts(request_payload)
end
end
Run the web service with command ruby webhook.rb
JSON fill be written to file events.txt

AWS dynamo DB service Proxy

I followed this tutorial to implement DynamoDB endpoint proxy service via API Gateway:
https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-as-a-proxy-for-dynamodb/
I manage to implement the PutItem and Query actions, but when I try to make a DeleteItem action i get the following error:
"message": "The provided key element does not match the schema"
That's my Get Method Integration:
{
"TableName": "Notes",
"Key": {
"myID": {
"S": "$input.params('ID')"
}
}
}
where myID is the primary key,
Any help will be greatly appreciated!

Dynamically set schedule in Azure Function

I have following function.json for my Azure function whose schedule is set to run at 9.30 daily. What I want is to dynamically set the schedule attribute of this json. This scenario arises when a client who is using my application enters date, on that date-scheduler should run.
{
"bindings": [
{
"name": "myTimer",
"type": "timerTrigger",
"direction": "in",
"schedule": "0 30 9 * * *" //Want to set dynamically
}
],
"disabled": false
}
Is this possible?
(Also note, I don't want to use Azure Scheduler due to budgeting reasons)
You could modify your function.json to fetch the cron expression from app settings.
"schedule": "%TriggerSchedule%"
Define TriggerSchedule in your appsettings. You could modify your appsettings dynamically and the function trigger would align to it.
Use Kudu API to change function.json
https://github.com/projectkudu/kudu/wiki/REST-API
PUT https://{functionAppName}.scm.azurewebsites.net/api/vfs/{pathToFunction.json},
Headers: If-Match: "*",
Body: new function.json content
Then send request to apply changes
POST https://{functionAppName}.scm.azurewebsites.net/api/functions/synctriggers
Or you can use Queue Trigger with "initialVisibilityDelay" messages. In this case you need to write your own code to implement a scheduler.
Its an old question but still relevant. I recently came across a similar issue. Azure function has a built in functionality that you can use. Its called Eternal orchestrations in Durable Functions (Azure Functions).
You can do something like
[FunctionName("Periodic_Cleanup_Loop")]
public static async Task Run([OrchestrationTrigger] IDurableOrchestrationContext
context)
{
await context.CallActivityAsync("DoCleanup", null);
// sleep for one hour between cleanups
DateTime nextCleanup = context.CurrentUtcDateTime.AddHours(1);
await context.CreateTimer(nextCleanup, CancellationToken.None);
context.ContinueAsNew(null);
}
More lnfo can be found at https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-eternal-orchestrations?tabs=csharp