I recently started using managed instance group with multi zone configuration. When i use GCE api to fetch instances for this instance groups zone is a required parameter. For a managed instance group with instances in multiple zone instance group does not belong to one zone. how do i fetch instances in this case?
What API or gcloud command are you using to list the instances? I guess you are using the instanceGroups.listInstances API, but for regional instance group, you need to use regionInstanceGroupManagers.listManagedInstances, or the corresponding gcloud command:
$ gcloud beta compute instance-groups managed list-instances instance-group-1 --region us-central1
NAME ZONE STATUS ACTION LAST_ERROR
instance-group-1-mk4j us-central1-b RUNNING NONE
instance-group-1-xnyk us-central1-c RUNNING NONE
instance-group-1-g23r us-central1-f RUNNING NONE
Note that this feature is still in beta.
Related
I am running a datapipeline every once a day and the pipeline creates a temporary EMR to run the activities. The EMR connects to a Mysql database and the IPs for the master and core nodes need to be whitelisted in the DB.
Is it possible to assign static or elastic IPs to nodes on EMR so that I don't have to whitelist the node IPs manually everytime the EMR is created?
Thanks in advance.
I was in the same situation and made a code for attaching the pre-obtained EIP to the master instance. But if you want to connect the AWS RDS, then you can simply allow the security group of EMR that is the best.
Below is what I used to fix the EIP for the master instance, not the others. In the case of the spark application with cluster mode, the master node has the driver session of the spark and only that is needed to be the whitelist.
#!/bin/bash
BOOL=`cat /emr/instance-controller/lib/info/instance.json | jq .isMaster`
if [ $BOOL == "true" ]
then
ID=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/instance-id`
aws ec2 associate-address --instance-id $ID --public-ip $1
fi
This script receives an argument that is the EIP what you want and I have put this code as a bootstrap action for the EMR. Be aware that the EMR should have the execution permission for associate-address.
I am using Ubuntu 18.04 on Google compute engine.
I am using the steps as shown in Google cloud documentation. My command is
sudo gcloud logging write "logname" "A simple entry"
The entry gets created but under the resource type as 'global'. However i want it to be created under resource name as compute engine.
I have tried setting logname as "projects/campuskudos-980/logs/appengine.googleapis.com%2Fvm.syslog" but that didn't work out
sudo gcloud logging write "logname" "A simple entry"
I want the logs to be created under GCE VM Instance resource type. So I can filter it out on stackdriver
Currently there’s no way to specify the resource type when using gcloud logging write command. As explained in the documentation for simplicity, this command makes several assumptions about the log entry. For instance, it always sets the resource type to global.
Right now, there are two ways to do that:
1- With the gcloud logging write command, use logname and specify something like projects/[PROJECT_ID]/logs/compute.googleapis.com. After that, using advanced filters on Stackdriver Logging as explained in the documentation, you can filter logs using an advanced filter to query all entries inside ‘compute.googleapis.com’.
For e.g.:
logName: (“projects/[PROJECT_ID]/logs/compute.googleapis.com”)
2- Call directly to API as explained in documentation specifying resource type as gce_instance.
Then that entry will appear under GCE VM Instance resource type on Stackdriver Logging.
I am trying to create an rds mysql read replica in a different vpc in the same region. This doesn't seem to work. I am getting the below error.
I am able to create a cross region read replica, here obviously the vpcs are different. But it works there and not within the same region.
Any idea why this could be the case?
The DB instance and EC2 security group are in different VPCs. The DB instance is in vpc-b40d62d3 and the EC2 security group is in vpc-3f6cc45b (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: 56d7eb7c-8cd7-490a-b979-ef678f4f6ed7)
This was asked earlier and the answer was not supported and use a custom solution. Asking again as that was a 3 year old post and cloud moves fast. :)
Cheers.
Tried this a week ago in RDS console but didn't work. I got the same error. But It is supported. At least in aws-cli version 1.16.
aws rds create-db-instance-read-replica \
--db-instance-identifier [yourmaindb] \
--source-db-instance-identifier [arn resource url of the source] \
--db-subnet-group-name [subnet in a different VPC] \
--vpc-security-group-ids [security group in a different VPC]
I was trying to add an IP# to my Google Compute Engine (RHEL7) instance, but I typed the invocation wrong:
sudo ifconfig eth0 1.2.3.4
The existing IP# on eth0 was 1.2.3.3, so that invocation changed my existing IP# to one that isn't known to anything else. And so I lost all connections (ssh, http, even ping) to the instance.
How do I recover from this mistake? Is there a gcloud or GCP Console method I can use, since I can't connect directly to the instance anymore.
Since the ifconfig was invoked from a shell, not reconfigured in any startup scripts (or anywhere else), just resetting the instance will reboot it and cause it to config its eth0 according to its startup scripts:
$ gcloud compute instances list
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
<instance-name> <instance-zone> <machine-type> <preemptible> <bad-internal-ip#> <external-ip#>
$ gcloud compute instances reset <instance-name>
For the following instance:
- [<instance-name>]
choose a zone:
[1] asia-east1-a
[2] asia-east1-b
[...]
Please enter your numeric choice: <N-of-instance-zone>
Updated [https://www.googleapis.com/compute/v1/projects/<project-name>/zones/<instance-zone>/instances/<instance-name].
$ gcloud compute instances list
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
<instance-name> <instance-zone> <machine-type> <preemptible> <default-internal-ip#> <external-ip#> RUNNING
After you enter your numeric zone it can take several seconds or a longer (but probably not more than 5 minutes) for the instance to restart.
Look around in the cloud platform console. You usually can change the external IP, then go the long way around - provided its instanced.
How can an autoscaling managed instance group be added to a target pool?
It's easy enough to add existing instances to a target pool via
$ gcloud compute target-pools create mypool --region us-central1
$ gcloud compute target-pools add-instances mypool \
--instances existing-instance1 existing-instance2 --zone us-central1-b
However, I want all the instances that appear in my autoscaling group to automatically be added to my target pool.
You can use gcloud compute instance-groups managed set-target-pools command to set the target pool for an existing manged instance group. You can refer to this link for more information.
There are four different types of resources in your setup:
instance is a virtual machine
a target pool is a pool of instances used only for the purpose of L3 (e.g. IP) level network load balancing
managed instance group is a group of instances, used among others as a target for your autoscaler
autoscaler looks at a managed instance group and adds/deletes instances in this group as appropriate according to load (and your policy)
To make sure that all the instances in your managed instance group (that is all the instances in your autoscaling group) are automatically in your target pool, you need to tell the managed instance group about the target pool.
As #Faizan correctly mentioned, the command to do it is:
gcloud compute instance-groups managed set-target-pools instance-group-name --target-pools your-target-pool
The help page for this command seems more useful than the online documentation:
gcloud compute instance-groups managed set-target-pools --help
Please note that this help page seems to be out of date though. Setting a new target pool now DOES apply to existing instances in the group (when using the API version v1 or later). It was not the case in the beta versions (v1beta2).