I just read in the google compute engine documentation that :
<<Note: Compute Engine is working with respective operating system communities and vendors to eventually convert all operating systems to automatically resize root partitions. As such, we recommend you occasionally check back to make sure this step is still needed for your operating system and over time, this step will be removed completely for all operating systems.>>
Did that mean (as I'm using Ubuntu that support automatic resizing) my compute will resize it's hard drive automatically after a certain level of disk space using. Actually I'm using 31% of my capacity, did the resizing occurs a a x% percent of disk space occupying ?
No. It means if you create a VM instance with a root persistent disk with more disk space than the original image, using an operating system that supports automatic resizing of the root partition, then your virtual machine automatically resize the partition to recognize the additional space and you won't need to manually repartition the disk.
In order to create a VM instance with a root persistent disk with more disk space than the original image, you will need use the following commands:
Create your root persistent disk:
$ gcloud compute disks create DISK_NAME --image IMAGE --size 100GB --zone ZONE
Then start a virtual machine instance using your root persistent disk:
$ gcloud compute instances create INSTANCE_NAME --disk name=DISK_NAME boot=yes --zone ZONE
Related
I have a GCE ubuntu 18.04 system that overflowed the / partition. Consequently, I can't start a remote shell to fix it. I created a new system from a snapshot, and stopped the old system.
I want to attach the old system disk to the new system, mount it, and delete a bunch of stuff so I can restart it.
However, when I edit the new system disk in the cloud console, the old system disk does not show up as a possible disk to attach. What do I need to do to the old system disk to both preserve it as a system disk, and allow attaching it to the new system?
What do I need to do to the old system disk to both preserve it as a
system disk, and allow attaching it to the new system?
Either the disk is still attached or the new instance is located in a different zone. Follow the steps below.
STEP 1:
Shut down your instance with the disk space problem. Login into the Google Cloud Console. Go to Compute Engine -> VM instances. Click on your instance and make note of the “Boot disk” name. This will be the first disk under “Boot disk and local disks”.
STEP 2:
Create a snapshot of the boot disk before doing anything further. While still in Compute Engine -> Disk. Click on your boot disk. Click on “CREATE SNAPSHOT”.
STEP 3:
Create a new instance in the same zone. A micro instance will work.
STEP 4:
Open a Cloud Shell prompt (this also works from your desktop if gcloud is set up). Execute this command. Replace NAME with your instance name (broken system) and DISK with the boot disk name and ZONE with the zone that the system is in:
gcloud compute instances detach-disk NAME --disk=DISK --zone=ZONE
Make sure that the command did not report an error.
STEP 5:
Now we will attach this disk to the new instance that you created.
Make sure that the repair instance is running before attaching the second disk. Sometimes an instance can get confused on which disk to boot from if more than one disk is bootable.
Go to Compute Engine -> VM instances. Click on your instance. Click Edit. Under “Additional disks” click “Add item”. For name enter/select the disk that you detached from your broken instance. Click Save.
STEP 6:
SSH into your new instance with both disks attached.
STEP 7:
Mount the second disk to the root file system.
Become superuser. Execute sudo -s
Execute df. Make sure that /dev/sdb1 is not mounted.
Create a directory for the mount point: mkdir /mnt/repair
Mount the second disk: mount /dev/sdb1 /mnt/repair
I have an application running on Container-Optimized OS based Compute Engine.
My application runs every 20min, fetches and writes data to a local file, then deletes the file after some processing. Note that each file is less than 100KB.
My boot disk size is the default 10GB.
I run into "no space left on device" error every month or so while attempting to write the file locally.
How can I track disk usage?
I manually checked the size of the folders and it seems that the bulk of the space is taken by /mnt/stateful_partition/var/lib/docker/overlay2.
my-vm / # sudo du -sh /mnt/stateful_partition/var/lib/docker/*
20K /mnt/stateful_partition/var/lib/docker/builder
72K /mnt/stateful_partition/var/lib/docker/buildkit
208K /mnt/stateful_partition/var/lib/docker/containers
4.4M /mnt/stateful_partition/var/lib/docker/image
52K /mnt/stateful_partition/var/lib/docker/network
1.6G /mnt/stateful_partition/var/lib/docker/overlay2
20K /mnt/stateful_partition/var/lib/docker/plugins
4.0K /mnt/stateful_partition/var/lib/docker/runtimes
4.0K /mnt/stateful_partition/var/lib/docker/swarm
4.0K /mnt/stateful_partition/var/lib/docker/tmp
4.0K /mnt/stateful_partition/var/lib/docker/trust
28K /mnt/stateful_partition/var/lib/docker/volumes
TL;DR: Use Stackdriver Monitoring and create an alert for DISK usage.
Since you are using COS images, you can enable Stackdriver Monitoring agent by simply adding the “google-monitoring-enabled” label set to “true” on GCE Instance metadata. To do so, run the command:
gcloud compute instances add-metadata instance-name --metadata=google-monitoring-enabled=true
Replace instance-name with the name of your instance. Remember to restart your instance to get the change done. You don't need to install the Stackdriver Monitoring agent since is already installed by default in COS images.
Then, you can use disk usage metric to get the usage of your disk.
You can create an alert to get a notification each time the usage of the partition reaches a certain threshold.
Since you are in a cloud, it is always the best idea to use the Cloud resources to solve Cloud issues.
Docker uses /var/lib/docker to store your images, containers, and local named volumes. Deleting this can result in data loss and possibly stop the engine from running. The overlay2 subdirectory specifically contains the various filesystem layers for images and containers.
To cleanup unused containers and images via command:
docker system prune.
Monitor it via command "watch"
sudo watch "du -sh /mnt/stateful_partition/var/lib/docker/*"
I am trying to migrate a couple of compute engine instances from Europe to Asia.
I need to do this because most of the users for those servers are in Asia and they get faster access to servers in their region. I am trying following command.
gcloud compute instances move asiawindows1 --zone europe-west1-d --destination-zone asia-east1-c
I get following error when I try this
Instances belonging to subnetworks cannot be moved interregionally.
What is the best way to move these servers without manually setting them up in the Asia zone?
Follow steps below in Google Cloud Platform
Create snapshot of your VM instance
Create disk with this snapshot
Create image with disk created
Create a new instance using the image created in steps above and change your zone
The command you are using is for moving across zones of same region and not across regions.
You can create an image of your instance and use the image to create a new instance in different region.
If the auto move is not happening, you can try the manual move of vm instance. First list all the disk to choose the right disk to take snapshot.
gcloud compute disks list
Identify the name of disk from the list and create a snapshot of that one
gcloud compute disks snapshot <disk_name> --snapshot-names <snapshot_disk_name> --zone <current_zone>
Now you can delete your instance in you are trying to move.
gcloud compute instances delete <instance_name> --zone <instance_zone>
Next step is to create a disk from the snapshot you have created in the new zone where you want to create the instance.
gcloud compute disks create <disk_name> --source-snapshot <snapshot_disk_name> --zone <new_zone_name>
Now, you have to create an instance with the disk you have just created mounted on it. Please choose the right machine type for your need, here am going to create an instance with f1-micro.
gcloud compute instances create <instance_name> --machine-type f1-micro --zone <new_zone>
--disk name=<disk_name>,boot=yes,mode=rw
That's it.
Now if you want you can delete the snapshot
gcloud compute snapshots delete <snapshot_name>
GCP Guide says Snapshot the root disk, create an image, and use the image for the new VM root disk.
How do I automatically restart a preemptible Google Compute Engine instance? I only have one instance that doesn't need 100% uptime but that I would like to restart once the data center becomes unloaded again. The instance/server that I'm trying to automatically restart has its own boot disk that I'd like to use each time it restarts.
You could try using Instance Group Manager to set up a pool of size 1. It will then try to re-create instances after they are preempted.
You should be aware that there is no guarantee that there is going to be capacity for your instance. As the docs say:
Preemptible instances are available from a finite amount of Compute Engine resources, and might not always be available.
You could create a f1-micro instance which is free for one instance per month in several data centers and create a cron job
*/10 * * * * /snap/bin/gcloud beta compute instances start --zone "yourzone" "yourinstance" --project "yourproject"
after you ran gcloud auth login once.
This will restart your instance every 10 minutes. Of course you can set this also to an hour or more. With a bit more scripting also things like exponential back off can be done.
If you'd like to restart it less frequently, you can use Instance schedules that's built in to the Google Cloud Dashboard.
https://cloud.google.com/compute/docs/instances/schedule-instance-start-stop
I have created a Google Compute Engine instance with CentOS and added some stuff there, such as Apache, Webmin, ActiveCollab, Gitolite etc.. etc.
The problem is that the VM is always running out of memory because the RAM is too low.
How do I change the assigned RAM in Google Compute Engine?
Should I have to copy the VM to another with bigger RAM? If so will it copy all the contents from my CentOS installation?
Can anyone give me some advises on how to get more RAM without having to reinstall everything.
Thanks
The recommended approach for manually managed instances is to boot from a Persistent root Disk. When your instance has been booted from Persistent Disk, you can delete the instance and immediately create a new instance from the same disk with a larger machine type. This is similar to shutting down a physical machine, installing faster processors and more RAM, and starting it back up again. This doesn't work with scratch disks because they come and go with the instance.
Using Persistent Disks also enables snapshots, which allow you to take a point-in-time snapshot of the exact state of the disk and create new disks from it. You can use them as backups. Snapshots are also global resources, so you can use them to create Persistent Disks in any zone. This makes it easy to migrate your instance between zones (to prepare for a maintenance window in your current zone, for example).
Never store state on scratch disks. If the instance stops for any reason, you've lost that data. For manually configured instances, boot them from a Persistent Disk. For application data, store it on Persistent Disk, or consider using a managed service for state, like Google Cloud SQL or Google Cloud Datastore.