I am trying to use google pubsub with cloud functions all of my things worked greatly but my messages are not being ackowledged
I have done these things:
1) Created a topic
2) Created a function
3) set trigger of function as google cloud pubsub
4) select topic for pubsub
5) set node version as 8
This is my default cloud function:
/**
* Triggered from a message on a Cloud Pub/Sub topic.
*
* #param {!Object} event Event payload.
* #param {!Object} context Metadata for the event.
*/
exports.helloPubSub = (event, context) => {
const pubsubMessage = event.data;
console.log(Buffer.from(pubsubMessage, 'base64').toString());
};
It will create 2 subscriptions for this topic one is pull and other to push
My cloud function is getting called but my messages are not getting acknowledged.
According to docs
Note: Cloud Functions acks the message internally upon successful function execution. For information on how to handle failures using retries, see Retrying Background Functions.
So functions should acknowledge automatically bit it is not working.
what is problem in this flow. what am I doing wrong?
Thanks
You do not need to explicitly create the subscription for a pubsub cloud function.
When deploying the cloud function and pointing it at a topic, the subscription is automatically created.
Create your-topic:
gcloud pubsub topics create your-topic
Deploy function pointing at your-topic (this automatically creates a subscription):
gcloud functions deploy your-function --region=us-central1 --trigger-topic=your-topic --runtime=python37 --entry-point=main
Run deploy command from the folder with your cloudfunction code. You can change the runtime and entry-point to match your use case.
Any secondary subscription you create has no consumers thus messages will not be acknowledged. Also, if your cloud function is failing it's messages may not be acknowledged.
If you want to try writing, deploying, and triggering a Background Cloud Function with a Cloud Pub/Sub trigger then the following documentation page is better.
https://cloud.google.com/functions/docs/tutorials/pubsub
I think you needs to trigger the function.
gcloud pubsub topics publish YOUR_TOPIC_NAME --message YOUR_MESSAGE
Related
i am quite new to the GCP world and i am asking here just to have some other informations about my issue.
I would like to create an HTTP Google Cloud Function that performs some operations over a certain project.
At the same way, i need to invoke this function from an external service (let's say from a Gsheet for which the SA that own the function could edit it).
I saw that there is GoogleAppScripts, and in particular the URLFetchApp service.
My idea is to call that function from that service.
In order to have authentication i saw that the library OAuth2 is needed, in this case, is it necessary to create a ClientID and a ClientSecret for the account that manage the function? If this is the case, how the AppScript could create the related service?
Thank you in advance.
You can query the identity token via google apps script and then use it to call the gcloud function.
The first thing we have to do is to have a cloud function and make sure you have permissions to invoke it (try executing it using another method).
After that, go to your Apps Script project manifest and set at least 2 specific scopes:
{
...
"oauthScopes": [
"openid",
"https://www.googleapis.com/auth/script.external_request"
],
...
}
openid allows us to get an identity token to call the function with and .../script.external_request allows us to use a fetch.
Then you can add the code:
const response = UrlFetchApp.fetch(
'https://gcloud-function-url',
{
muteHttpExceptions: true,
headers: {
'Authorization': `Bearer ${ScriptApp.getIdentityToken()}`
}
}
)
// use response
As documented in the reference, ScriptApp.getIdentityToken() returns the the OpenID token that we need.
References
Authenticating Developers, Functions, and End-users
(Google Cloud guides)
UrlFetchApp.fetch(url, params) (Google Apps Script reference)
ScriptApp.getIdentityToken() (Google Apps Script reference)
This works as stated in #Martí 's answer, but I spent a very frustrating day having it not work because I missed a tiny but crucial step: You need to redeploy the Cloud Function after making the Apps Script, otherwise it won't have the Client ID validated! The Apps Script is assigned this ID when you associate it with the same GCP Project, so redeploy the Function after that.
If I delete a google cloud function and redeploy the same function a day later will I have lost the pub/sub messages that trigger it for that day?
It depends how you have deployed your Cloud Functions
HTTP Functions: You deployed your Cloud Functions with an HTTP trigger. In this case, you have to create a PubSub Push subscription manually that call the Cloud Functions when a message arrived.
If you delete your Cloud Functions, the Pubsub subscription continue to live and to receive the messages. They are kept up to their expiration deadline. The push HTTP call fails in error continuously because the Cloud Functions no longer exists.
If you redeploy the Cloud Functions, on the same project, with the same name, the HTTP url will again exist and the messages will be delivered, without lost.
Background Functions: You have deployed your function with a trigger topic. In this case a PubSub subscription is created automatically with the function is created
If you delete the Cloud Functions, the PubSub subscription, created automatically, will be deleted automatically. So, the existing messages and the new ones will be lost, because there is no subscription to accept and store them.
I have a Pub/Sub topic where I publish messages and that pubsub topic is subscribed by a Google Cloud Function which sends the email to the customer using mailgun API.
This works fine so far, now I got the requirement that I have to send the email with the file attachment.
I can easily do this using Google Cloud Function using HTTP trigger but how to do it using a Pub/Sub topic?
Check the given image.
Note: The file size could be more than 10mb in my case.
First of all if I understand your goal correctly, PubSub alone will never be enough as the actual work will happen at the function.
To achieve the above you need a cloud function with Pub/Sub trigger.
Once the function is called, you need to either fetch the attachment from a Google Cloud Bucket, or download it from a another API/source you have. You'll have to download it inside the /tmp directory which is used to store temporary files.
Once completed, then you can use mailgun or other tools of your choice. Remember the attachment is inside the /tmp directory and not ./
Using mailgun or any other method does not change anything on the overall approach. For example if you were to use Gmail API, you'd add this to your cloud function for the final step.
How can I securely call a Google Cloud Function via a Google Apps Script?
✅ I have a Google Cloud Function, which I can access at https://MY_REGION-MY_PROJECT.cloudfunctions.net/MY_FUNCTION, and which I would like to allow certain users to invoke via an Apps Script.
✅ To secure the Cloud Function, I have set Cloud Function Invoker to only include known email (e.g. USER#COMPANY.com, where this is a valid Google email).
✅ I am able to successfully invoke the Cloud Function via curl, while logged into gcloud with this email, by running: curl https://MY_REGION-MY_PROJECT.cloudfunctions.net/MY_FUNCTION -H "Authorization: Bearer $(gcloud auth print-identity-token)".
✅ I have granted the following oauthScopes in my Apps Script's manifest:
"https://www.googleapis.com/auth/script.external_request"
"https://www.googleapis.com/auth/userinfo.email"
"https://www.googleapis.com/auth/cloud-platform"
⛔️ However, when I attempt to invoke the Cloud Function via a Google Apps Script, while logged in with the email USER#COMPANY.com, I am unable to invoke it and instead returned a 401. Here is how I have attempted to invoke the Cloud Function:
const token = ScriptApp.getIdentityToken();
const options = {
headers: {'Authorization': 'Bearer ' + token}
}
UrlFetchApp.fetch("https://MY_REGION-MY_PROJECT.cloudfunctions.net/MY_FUNCTION", options);
ℹ️ I have also tried the following:
Using ScriptApp.getOAuthToken()
Adding additional oauthScopes, e.g. openid.
Creating an OAuth Client ID with https://script.google.com set as an Authorized Javascript origin.
Deploying the Apps Script.
Crying out to the sky in utter, abject despair
I struggled very much authenticating from Apps Script to invoke a Cloud Run application and just figured it out, and I believe it's similar for calling any Google Cloud application including Cloud Functions. Essentially the goal is to invoke an HTTP method protected by Google Cloud IAM using the authentication information you already have running Apps Script as the user.
The missing step I believe is that the technique you're using will only work if the Apps Script script and Google Cloud Function (or Run container in my case) are in the same GCP project. (See how to associate the script with the GCP project.)
Setting it up this way is much simpler than otherwise: when you associate the script with a GCP project, this automatically creates an OAuth Client ID configuration to the project, and Apps Script's getIdentityToken function returns an identity token that is only valid for that client ID (it's coded into the aud field field of the token). If you wanted an identity token that works for another project, you'd need to get one another way.
If you are able to put the script and GCP function or app in the same GCP project, you'll also have to do these things, many of which you already did:
Successfully test authentication of your cloud function via curl https://MY_REGION-MY_PROJECT.cloudfunctions.net/MY_FUNCTION -H "Authorization: Bearer $(gcloud auth print-identity-token)" (as instructed here). If this fails then you have a different problem than is asked in this Stack Overflow question, so I'm omitting troubleshooting steps for this.
Ensure you are actually who the script is running as. You cannot get an identity token from custom function in a spreadsheet as they run anonymously. In other cases, the Apps Script code may be running as someone else, such as certain triggers.
Redeploy the Cloud Function as mentioned here (or similarly redeploy the Cloud Run container as mentioned here) so the app will pick up any new Client ID configuration. This is required after any new Client ID is created, including the one created automatically by adding or re-adding the script to the GCP project. (If you move the script to another GCP project and then move it back again, it seems to create another Client ID rather than reuse the old one and the old one will stop working.)
Add the "openid" scope (and all other needed scopes, such as https://www.googleapis.com/auth/script.external_request) explicitly in the manifest. getIdentityToken() will return null without the openid scope which can cause this error. Note to readers: read this bullet point carefully - the scope name is literally just "openid" - it's not a URL like the other scopes.
"oauthScopes": ["openid", "https://...", ...]
Use getIdentityToken() and do NOT use getOAuthToken(). According to what I've read, getOAuthToken() returns an access token rather than an identity token. Access tokens do not prove your identity; rather they just give prove authorization to access some resources.
If you are not able to add the script to the same project as the GCP application, I don't know what to do as I've never successfully tried it. Generally you're tasked with obtaining an OAuth identity token tied to one of your GCP client ids. I don't think one app (or GCP project) is supposed to be able to obtain an identity token for a different OAuth app (different GCP project). Anyway, it may still be possible. Google discusses OAuth authentication at a high level in their OpenID Connect docs. Perhaps an HTML service to do a regular Google sign-in flow with a web client, would work for user-present operations if you get the user to click the redirect link as Apps Script doesn't allow browser redirects. If you just need to protect your service from the public, perhaps you could try other authentication options that involve service accounts. (I haven't tried this either.) If the service just needs to know who the user is, perhaps you could parse the identity token and send the identifier of the user as part of the request. If the service needs to access their Google resources, then maybe you could have the user sign in to that app separately and use OAuth generally for long term access to their resources, using it as needed when called by Apps Script.
The answer above is very good. But since I am new with this I still had to spend a lot of time trying to figure it out.
This worked for me:
Apps Script code:
async function callCloudFunction() {
const token = ScriptApp.getIdentityToken();
const options = {
headers: {'Authorization': 'Bearer ' + token}
}
const data = JSON.parse(await UrlFetchApp.fetch("https://MY_REGION-MY_PROJECT.cloudfunctions.net/MY_FUNCTION", options).getContentText())
return data
}
Make sure that in project config you have the same project where your function is created.
After that, you can add the emails of the users you want to access the script on the Permission section in the function.
And as #alexander-taylor mentioned as well, make sure to add the scopes to your manifest file. You can make the manifest visible from the configuration tab in apps script. It took me some time to get that too.
Thanks to your comment you can do 2 things. But before, you have to know that you can't (or I least I never achieve this), create a valid identity token for being authenticated by Cloud Function and Cloud Run with a user credential. I opened a question on this
But you can call Google Cloud API with user credential! So
You can use the function test call API. The quotas limit you to 16 calls per 100 minutes (of course, it's design for test!)
You can publish a message into PubSub and plug your function on it. In this pattern your call is asynchronous.
I am trying to create a custom Google Sheets function that pulls data from the Analytics Reporting V4 API. When I run this function from the App Scripts console, it successfully returns data.
When I run the function from within my spreadsheet, I get the following error:
API call to analyticsreporting.reports.batchGet failed with error: Request is missing required
authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication
credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. (line 59).
Here is my function defined in the Apps Script Editor (again, this works fine when I press the run button from here):
And here is my error-producing spreadsheet implementation:
Is there a way that I can make this work without having to use OAuth credentials? From my understanding, the benefit of using Advanced Google Services is the ability to avoid this authentication flow, and I would like to take advantage of it.
In the documentation:
If your custom function throws the error message You do not have permission to call X service., the service requires user authorization and thus cannot be used in a custom function.
To use a service other than those listed above, create a custom menu that runs an Apps Script function instead of writing a custom function. A function that is triggered from a menu will ask the user for authorization if necessary and can consequently use all Apps Script services.
Using Apps Script Services