I want to be able to query the external IP address of a GCE instance when the instance starts up. I'm planning to use that to fix up some configs which are copied to multiple similar instances. Is there a way to automatically discover an instance's external IP(s) or other properties from the instance itself? I see there are some things you can query with the gcloud tool, but for that you have to know the instance name, and it's not clear where to get that from.
See Querying metadata in GCE public documentation. For example, for the instance's external IP:
curl http://metadata/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip/ -H "Metadata-Flavor: Google"
This command will query the instance's private metadata server. Another option is configuring the instance's service account with the right scopes as described at Preparing an instance to use service accounts in the public documentation. This way, gcloud command can be used directly in the instance to get information from the project without authentication.
Related
I am writing an AWS Lambda function to connect to a MySQL instance running on EC2. I have associated the Lambda function with the same subnet and security group that the EC2 is configured in. I checked the Lambda function's IAM roles and that has AWSLambdaVPCAccessExecutionRole policy attached to it. However I am still not able to connect to the MySQL instance.
I tried allowing traffic from anywhere and that worked but now I am not sure how to connect to the MySQL instance with stricter security rules.
I am using Kotlin to write my lambda function and using serverless to deploy changes to lambda.
I have tried every possible solution available online to make this happen but I haven't had any positive results yet.
You have associated the Lambda function with the same security group. But just that would not do. You also need to add an ingress rule to allow traffic to the security group from itself. Basically, you need to self reference the security group.
Add a rule to allow traffic on the mysql port from sg-xxxxxxxx.
I have jar file of springboot and I'm running on compute engineVM
And I also connect SQL-client but what address of mysql should I give in spring boot
I assume you are using GCP's hosted mysql? (Cloud SQL).
If so, then if you are connecting to it via cloud sql proxy, which is running on the same machine, then you just use localhost. The proxy should know the way to the server from there, assuming that you've configured the instance name and project/etc. correctly.
Otherwise, without the proxy, you can use your SQL instance's public IP address, which you can see on the list of running instances when you select the SQL page.
In the second case (using the actual IP address) keep in mind that GCP probably wont let the VM running your application through the firewall to the SQl instance directly. To work around this, you'd have to list your VM's IP address in the Authorized Networks section of the SQL entry (click on your SQL instance in the list and select the Authorization tab). Again, in this case, you need to keep in mind that your VM's IP address is ephemeral by default (unless you made and effort to make it permanent). So if you restart your VM, the above Authorization will no longer make sense. So make sure you make your VM's IP address permanent.
I am stuck at one thing regarding CloudSQL.
I have my WordPress app running on GCE and I create Instance Group so I will utilise the AutoScaler.
for Db, I am using CloudSQL.
Now point where is stuck is the "Authorise network" in CloudSQL as it accepts only IPV4 Public IP.
How do I know when autoscaling happen what IP will attach to Instance so my instance will know where the DB is?
I can hard code the CloudSQL IP as a CNAME but from CloudSQL Side I am not able to figure it out how to provide access. I can make my DB access all open
If you can let me know what will be the point which I am missing.
I used cloudsql proxy also but that doesn't come with Service in Linux ... I hope you can understand my situation. Let me know if any idea you like to share on this.
Thank you
The recommended way is to use the second generation instances and Cloud SQL Proxy, you’ll need to configure the Proxy on Linux and start it by using service account credentials as outlined at the provided link.
Another way is to use startup script in your GCE instance template, so you can get your new instance’s external IP address and add it to a Cloud SQL instance’s authorized networks by using gcloud sql instance patch command. The IP can be removed from the authorized networks in the same way by using shutdown script. The external IP address of GCE VM instance can be retrieved from metadata by running:
$ curl "http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip" -H "Metadata-Flavor: Google".
Need to submit dataflow job from an existing GCE VM in google cloud, learned that there has to be one service account with proper scope to be attached to that VM when the VM is created, what if VM already existed? how to attach a service account to an existing vm?
According to the GCE docs you cannot change the attached service account after instance creation:
After you have created an instance with a service account and specified scopes, you cannot change or expand the list of scopes.
See
https://cloud.google.com/compute/docs/authentication#using
for more details.
However if you don't want to recreate your VM you should be able to create a service account and authenticate to that using a private key, as described in the following:
https://developers.google.com/identity/protocols/OAuth2ServiceAccount
This is likely less convenient than the using a VM service account because you'll need to manage the private key and authentication yourself.
I have deployed a hadoop cluster on google compute engine. I then run a machine learning algorithm (Cloudera's Oryx) on the master node of the hadoop cluster. The output of this algorithm is accessed via an HTTP REST API. Thus I need to access the output either by a web browser, or via REST commands. However, I cannot resolve the address for the output of the master node which takes the form http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091.
I have allowed http traffic and allowed access to ports 80 and 8091 on the network. But I cannot resolve the address given. Note this http address is NOT the IP address of the master node instance.
I have followed along with examples for accessing IP addresses of compute instances. However, I cannot find examples of accessing a single node of a hadoop cluster on GCE, that follows this form http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091. Any help would be appreciated. Thank you.
The reason you're seeing this is that the "HOSTNAME.c.PROJECT.internal" name is only resolvable from within the GCE network of that same instance itself; these domain names are not globally visible. So, if you were to SSH into your master node first, and then try to curl http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091 then you should successfully retrieve the contents, whereas trying to access from your personal browser will just fail to resolve that hostname into any IP address.
So unfortunately, the quickest way for you to retrieve those contents is indeed to use the external IP address of your GCE instance. If you've already opened port 8091 on the network, simply use gcutil getinstance CLUSTER_NAME-m and look for the entry specifying external IP address; then plug that in as your URL: http://[external ip address]:8091.
If you turned up the cluster using bdutil, a more involved but nicer way to access your cluster is by running the bdutil socksproxy command. This opens a dynamic-port-forwarding SSH tunnel to your master node as a SOCKS5 proxy, so that you can then configure your browser to use localhost:1080 as your proxy server, make sure to enable remote DNS resolution, and then visit your browser using the normal http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091 URL.