From my workstation I can fire templated Dataflow jobs with the gcloud dataflow jobs command. The required authorization to insert a new job come from my workstation where I'm logged in.
On the Compute Engine instance I rely on it's service account. The one with (number)-compute#. Within the AIM section I enabled Dataflow/Dataflow Admin, Dataflow/Dataflow Developer and Dataflow/Dataflow Worker for this service account to be safe.
I even added Cloud Dataflow Service Agent when I came across that one.
Then I try to start a Dataflow from the command line but I get an error about insufficient authentication scopes: ERROR: (gcloud.dataflow.jobs.run) PERMISSION_DENIED: Request had insufficient authentication scopes.
If I do a gcloud config auth and login with my personal account, of course, it works.
Somehow I'm missing the proper permissions to set to the applied service account.
Is there a guideline I missed? Can somebody please point me into the right direction?
The error message indicates that the instance does not setup access scope properly. To launches a job from a GCE VM, the VM must have compute.read-only, compute, or cloud-platform scope for the project.
The way to verify it is using the command "gcloud compute instances describe --zone=[zone][instance]" and look for "scopes".
This document and this existing question may provide useful guidelines for you.
Related
Keep getting this error:
Error, failed to create instance group-database-instance: googleapi: Error 403: Request had insufficient authentication scopes.
More details:
Reason: insufficientPermissions, Message: Insufficient Permission
I have added a service account with editor permissions to use all gcp resources and added directed terraform to a credentials file generated.
Would this be an error in the code or something else?
Based on the error message you have provided and the task you would like to accomplish, it would seem that you might need to add a scope when creating your instance.
To use the Google Kubernetes Engine API for a GCE virtual machines, you will need to add the Cloud Platform Scope ("https://www.googleapis.com/auth/cloud-platform") to your VM when it is created.
Additionally, if you are using the gcloud command-line, you can follow along with something like:
gcloud compute instances create NAME --scopes=https://www.googleapis.com/auth/cloud-platform
If you are using the Cloud Console UI, when you are creating a VM instance, look for the "Identity and API access" section, and select "Allow full access to all Cloud APIs".
I'm trying to set up read/write access to a Cloud Storage bucket from a GCE instance, using a service account, but don't get the permissions. I have done the following:
Created service account, let's say 'my-sa'
Created a bucket, let's say 'my-bucket'
In IAM console for my project, assign role 'Cloud Storage admin' to service account
Created a new GCE instance via the console, assigned to service account 'my-sa'. Access scope is then automatically set to cloud-platform
Connect to instance using gcloud compute ssh as my user (project owner)
Run gsutil ls gs://my-bucket
Expected behaviour: get list of items in bucket
Observed behaviour:
gsutil takes about 5 seconds to think, then gives:
AccessDeniedException: 403 my-sa#my-project.iam.gserviceaccount.com does not have storage.objects.list access to bucket my-bucket.
Things I've tried:
gcloud auth list on the instance does show the service account, and shows it as being active
I've added more permissions to the service account (up to project owner), doesn't make a difference
I also can't use other permissions from the instance. When I give Compute Engine Admin role to the service account, I can't run gcloud compute instances list from the instance
I've removed the .gsutil dir to make sure the cache is cleared
With the default Compute Engine service account, I can list the buckets, but not write (as expected). When I add the Cloud Storage read/write access scope from the console, I can also write
I really don't have a clue on how to debug this anymore, so any help would be much apprreciated
I have a Linux VM on Google Compute Engine that I am accessing via SSH. It works just fine, but when I go to the Cloud Console, it asks me if I want to create a new VM as if I have none. I know I'm on the right account because it shows my billing balance has gone down.enter image description here Where did my server go?
It is weird. But it is important to make a differentiation that is not obvious once you start using Google Cloud Platform. The credentials you are using to access the Platform ( your email or a service account), the projects where an entity that any resource must be attached to and the billing account that is the payment profile that can have several projects associated.
In that case you could be in a different project, that is associated to the same billing account.
To check you can the project where your machine is, in the shell
Gcloud compute instances list
Here you will see the instances in your actual project. If nothing appears, reset gcloud configuration.
gcloud init
And change the project.
Our team is trying to troubleshoot an issue we have been encountering with service accounts. The service account we are using is able to create a disk and IP address, however an error is thrown when an instance request is created. All resources can be listed (ie. networks, snapshots, etc.). I have attached a small console snippet below.
The service account is successfully authenticated with JSON key given to me. I have tried altering permissions of the service account and created a new key.
Any assistance is greatly appreciated.
Created [https://www.googleapis.com/compute/v1/projects/<PROJECT>/zones/asia-east1-c/disks/dev-josh-ui-test-08].
Created [https://www.googleapis.com/compute/v1/projects/<PROJECT>/regions/asia-east1/addresses/dev-josh-ui-test-08-ip].
ERROR: (gcloud.compute.instances.create) Some requests did not succeed:
- The resource '<ID>-compute#developer.gserviceaccount.com' of type 'serviceAccount' was not found.
I was able to get the exact error provided:
The resource '-compute#developer.gserviceaccount.com' of type 'serviceAccount' was not found.
by deleting my default compute service account and attempting to create an instance through the Cloud Shell, so I assume this is the issue.
If the default compute service account was somehow deleted, if has been less than 30 days, you can restore it using: gcloud beta iam service-accounts undelete [ACCOUNT_ID]
https://cloud.google.com/iam/docs/creating-managing-service-accounts#undeleting
After this, you will have to go into https://console.cloud.google.com/apis/dashboard and disable and re-enable the compute engine API. This will take a few moments, but after the GCE API is re-enabled you should be able to create VMs through the Cloud Shell again and I was able to reproduce this.
On https://console.cloud.google.com/apis/dashboard disable the "google compute engine API" and after enable it again.
The enabling also creates some additional setup that is needed to use the API. Those resources could have been deleted by accident beforehand.
You might need to have some patience and wait a minute or two between disabling and enabling.
I have error when I run google cloud command, this is the error message which I get
$ gcloud compute instances list
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.list) Some requests did not succeed:
- Access Not Configured. The API is not enabled for your project, or there is a per-IP or per-Referer restriction configured on your API key and the request does not match these restrictions. Please use the Google Developers Console to update your configuration.
I have two machine running and I already updated to new version.
I don't know if you are encountering this problem, but if you specify the project name instead of the project ID when you do "gcloud config set project " then you will currently get the "Access Not Configured" error. I've pointed it out on the #gcloud IRC so hopefully it gets fixed. There may be other issues like this so it is best to ensure your parameters are sane.
If your project hasn't been marked for abuse and/or deletion, you have to enable 'Google Compute Engine' API in the Developers Console to solve the problem.
Configure your project using following command
gcloud config set project <project-id>
This is the exact API we need to enable in order to get rid of this error
Compute Engine API