I've set up an alerting policy in StackDriver on the "instance/uptime" metrics. To alert when it's less 1 for 1 minute.
Then I deleted the instance, and had no alerts.
Is it so because at the following time window the data isn't 0 rather none, so no alerts are sent?
I reproduced your situation
If the instance is deleted, there's no alerts being generated, this is expected behavior since the resource (VM) doesn't exist anymore.
Uptime alerts are generated when only VM instances are up and running.
It seems you are trying to send alert when instances are deleted, if this is the case, this can be done with log-based metrics in Stackdriver Logging which can be used in Stackdriver Monitoring to generate alerts. The following links will assist in this regards Creating Counter Metrics and Logs-based metrics interface
Related
In the documentation, the firebase resize extension has the piece of code
exports.onimageresized = onCustomEventPublished(
"firebase.extensions.storage-resize-images.v1.complete",
(event) => {
logger.info("Received image resize completed event", event);
// For example, write resized image details into Firestore.
return getFirestore()
.collection("images")
.doc(event.subject.replace("/", "_")) // original file path
.set(event.data); // resized images paths and sizes
});
only when I check for that log in the function logs, there is nothing, only a create operation log(for when the function was deployed). Weird thing is the function still writes to firestore, I actually thought it was not firing.
Where are the eventarc logs read from, I cannot seem to find them
The following types of audit logs are available for Eventarc:
Admin Activity audit logs Includes "admin write" operations that
write metadata or configuration information. You can't disable Admin
Activity audit logs.
Data Access audit logs Includes "admin read" operations that read
metadata or configuration information. Also includes "data read" and
"data write" operations that read or write user-provided data.
To receive Data Access audit logs, you must explicitly enable them.
Eventarc audit logs use the service name eventarc.googleapis.com.
Eventarc audit logs use the resource type audited_resource for all audit logs.
You can view audit logs in Cloud Logging by using the Google Cloud console, the Google Cloud CLI, or the Logging API.
To view directly using the Cloud console follow the below steps:
In the Google Cloud console, go to the Logging> Logs Explorer page
Select an existing Cloud project, folder, or organization.
In the Query builder pane,In Resource type, select the Google Cloud
resource whose audit logs you want to see and in Log name, select
the audit log type that you want to see.
If you're experiencing issues when trying to view logs in the Logs Explorer, see the troubleshooting information.
Also check the documentation for functions calling eventrac and supported events.
When I create a Google Cloud Function, I am offered to set a "Trigger":
and the "Maximum number of instances":
What happens if a Pub/Sub triggered Cloud Function faces periods when it receives more messages than it can instantly process?
Concrete example: I am sending 200 strings to the Cloud Function per minute, for 5 minutes. One instance can process 10 strings in a minute, and the "Maximum number of instances" is 10, so in total, 100 strings per minute is processed. What will happen to the other half of the 200 strings, will they "wait" in the Pub/Sub topic until they are processed, or will these inputs get lost?
What happens if a Pub/Sub triggered Cloud Function faces periods when it receives more messages than it can instantly process?
Initially, this would result to some messages to not process (UNACK will be sent), and will be left behind. However, to be able to work around with this, you just need to enable the retry policy on the function.
Cloud Functions guarantees at-least-once execution of an event-driven function for each event emitted by an event source. However, by default, if a function invocation terminates with an error, the function will not be invoked again, and the event will be dropped. When you enable retries on an event-driven function, Cloud Functions will retry a failed function invocation until it completes successfully, or the retry window (by default, 7 days) expires.
You can enable retries by providing the --retry flag when creating the function via the gcloud command-line tool or checking the "Retry on failure" box when creating via the Cloud Console.
To update the subscription to use a retry policy, find the name of the subscription created by Cloud Functions in the Cloud Console Pub/Sub section or use the gcloud pubsub subscriptions update command with the necessary flags.
The messages/events just queue up in the subscription until they get processed. No need to specify retry behaviour with --retry.
I have deployed simple PubSub Cloud Function trigger using this tutorial: https://medium.com/#milosevic81/copy-data-from-pub-sub-to-bigquery-496e003228a1
For test I pushed large (over 8MB) message to PubSub topic.
As a result Cloud function returned the following error message to the log: Function execution could not start, status: 'request too large'
The issue is, that Cloud Function started to fire up constantly producing constant resource usage and log messages. It stopped only after I manually purged the related PubSub topic.
Is there a mechanism/configuration to prevent such behavior? Ideally PubSub message should not be picked again after Cloud Function trigger execution.
You reached the quotas of Cloud Functions
Max uncompressed HTTP request size -> 10MB
One solution is to use Cloud Run (the quotas is higher, 32Mb)
For this, you need several changes
Convert your Cloud Functions in Cloud Run. I wrote an article (not dedicated to this but you have an example in Python), and I presented this at GDG Ahmedabad last month, in GO this time
Create a push subscription on your PubSub topic and use the Cloud Run HTTPS endpoint in the "push" HTTP field
Cloud Run can handle up to 80 concurrent requests on 1 instances, Cloud Functions only one. Because your request are "big" it might cause memory issues if you process too many request in the same instance. You can control this with Cloud Run with the --concurrency param. Set it to 1 to have the same behavior as CLoud Functions.
I am using Cloud Functions on Google Cloud Platform. I have set up a Stackdriver alerting policy to send me notifications when those functions exceed an execution time threshold.
I would like to specifically get the logs for function instances that violate this policy. But when I click on the "Logs" link on the Policy Violation page of the Stackdriver user interface, it shows me all the logs for that function.
How can I filter Cloud Function logs to only get logs of instances which violate a Stackdriver alerting policy?
Update:
To clarify, my intention is to get the text logs of instances that violate a policy, rather than a summary metric.
If you want to filter on Cloud Function logs of an instance which violate a stackdriver alerting policy, then you need to create a "custom metric" (Stackdriver > Logs-based metrics > Create Metric) to get the execution time of a particular function.
You can find more detailed information in this article, Getting Google Cloud Functions times in Stackdriver.
However, the custom metric will parse the logs and give Stackdriver a number you can put in a chart. After that, you can set an alert for that chart.
how to log the response size ,request size, error message from API management instance?
If yes then how can I fetch the data from it.
There is an inbuilt "log-to-event-hub" policy that you can use to send basically any information that exists on the context object (meaning the request/response + a bit more) to an event hub. From there you can use any regular method for processing the events
How to log events to Azure Event Hubs in Azure API Management.
Use Azure Monitor to configure diagnostic logs from ApiManagement to either Storage, Eventhub or Log Analytics. These logs have the data you are looking for.
I would start with free tier of Log Analytics for easy querying, dashboards and alerting. Refer this.
For more custom logging, you can use log to event hub policy. Refer this blog.