Customizing a GCE Ubuntu VM image - google-compute-engine

I have a Google Cloud Platform account that I access from a VirtualBox VM. I am using the Google Compute Engine for a project that I am currently working on, and I had to create a custom image based on the Ubuntu 14.04 image that's available there.
I made changes to the Ubuntu image by ssh'ing into an Ubuntu 14.04 instance, (from my Vbox VM terminal) installing the Matlab compiler runtime, and downloading some other files that I needed. I created the custom image by following the steps according to the documentation.
However, now the changes I made are only available to me when I SSH from my Vbox VM terminal. I need to be able to run a certain matlab program Via startup scripts, how can I make it so that all users using this image have access to the customizations I made? Is there a way I can do this without having to make the edits by ssh'ing from the developers console and redoing all the changes?
EDIT: I don't think I was very clear so Ill give some examples. say my Google account is alexanderlang. When I ssh into an instance created from my custom image from the developers console, bash prompt looks like:
alexanderlang#myinstance $
My Vbox username is alex, and when I ssh into the same instance from my vbox terminal, bash prompt looks like:
alex#myinstance $
alex#myinstance can run matlab programs, but alexanderlang#myinstance cannot. I'm talking about the same instance, created from the same image. I think this might have something to do with the ssh keys for my custom image, but I don't know how to change or remove those keys.

When you connect to your VM instance via ssh by using either Developers Console or gcloud, the user account is dynamically created (if it doesn't already exist) by setting metadata on the VM. The question is: how does each tool choose your username?
When you use Google Developers Console, the only information it knows about you is your Google Account name, so it uses that, e.g., <first-name>_<last-name> or similar.
When you connect to your instance via gcloud, it knows the value of $USER so it uses that instead.
Note that in either case, your account has passwordless sudo access, so if you want to switch from one account to the other, you can run:
sudo su alex
while logged in as alexanderlang and then you have access to all the programs that alex does.
Similarly, you can run:
sudo su alexanderlang
while logged in as alex to do the reverse.
Startup scripts run as root. To run commands as another user, you need to do two things:
change to that username
run commands as that user
sudo su alex will create a new shell and hence ignore the rest of the script (until you manually exit the user shell, which is not what you want).
You can use sudo su alex -c 'command to run' but since what you want to run is a complex script, you need to first save the script to a file, and then run it.
Your options are:
pre-create the shell script to run
dynamically generate it from the startup script
Doing (1) is easy if the script never changes. For frequently-changing scripts (and it sounds like, many dynamically created VMs), you want to use option (2).
Here's how to do this in a startup script:
cat > /tmp/startup-script-helper.sh <<EOF
# ... put the script contents here ...
EOF
sudo su alex -c '/tmp/startup-script-helper.sh'

You can use Packer to create a derived image from a stock GCE VM image. Packer will let you do the following very easily:
boot a GCE VM using an image you specify
run some customization step, e.g., shell script, or Chef/Puppet/etc.
save the resulting image in your Google Cloud Platform project
Then, you can boot any number of new VMs using your newly-created image.
Note that since your VM image will be stored on Google Cloud Storage, you will be charged for the space it uses. Current pricing for Google Cloud Storage standard class is USD $0.026 / GB / month. A typical VM image should be less than 1GB.
You can see a complete example of how I used Packer to build VMs and pre-installed Ambari on it via my GitHub repo.

Related

How do you change the public key of a Oracle Cloud Instance?

I accidentally deleted my public and private key and had to generate new SSH keys due to not being able to restore the keys (and not having a backup anywhere). How do I change the public SSH key then of my Oracle Cloud instance?
Terminating the instance and remaking it isn't an option, and I've tried looking online but wasn't able to find much. Any help would be appreciated.
Thanks
Some background
Found a solution! Just so people are aware, there are methods online that involve connecting to the machine via VNC, but for me personally it felt very trial-an-error, when pressing buttons at the wrong time, and it ended up not working properly (VNC didn't display recovery mode for me, just a blank screen after selecting it).
Summary
This guide involves: Creating another machine (as incl. in free tier anyway), detaching the boot volume drive from the machine and attaching it to the machine just created, to do editing to change the keys over, then attaching the drive back up.
Create another VPS (Oracle have them incl. > free tier)
I deleted one of my other VPS' in the Oracle panel (that was a free machine - as I didn't need it and wasn't using it) and created it again anew (I made sure to delete the old boot volumes before continuing).
(This solution is assuming your using Ubuntu 20.04, but this will probably work for other OS's as well)
Basically from there,
I powered off the machine I wanted to change my SSH key of.
After fully being powered off, just detach the boot volume from the VPS, and attached it as a block volume to the machine just created.
Login to the machine via SSH, and run the connection commands by hitting the three dots (image below) and viewing the connection commands, to connect the drive up.
Editing files on the drive & mounting process
Then by running blkid (or sudo fdisk -l for a more friendly view)
you're able to see what drives are available for mounting. So then you just make a folder and simply type:
sudo mount [drive path e.g. /dev/sdb] [folder path e.g. ./drive]
Edit the file at /home/ubuntu/.ssh/authorized_keys, or however your machine is configured (Oracle by default disallows root, but if you've edited your configuration it's up to your end).
Then, simply go to the relevant path to be able to unmount the drive, umount [folder path e.g. ./drive]
Run the disconnect commands for the drive from the panel.
Then, simply detach the drive from your other machine and reattach it back to the original machine. Wait till it's fully attached and then start the machine again.
You can create a console connection, connection to it, then reboot the instance (through OCI console), and get to GRUB in the console connection... a few more steps and you can upload a new ssh key: https://docs.oracle.com/en-us/iaas/Content/Compute/References/serialconsole.htm

2 factor authentication (2 step verification) with Google compute engine

Is there a possibility to enable 2 factor authentication (or 2 step verification a-la Google terminology) for Google compute engine?
I'm interested in protecting my VMs, cloud storage and the developers console.
I've tried using the Google Authenticator (libapm) referring to this article Securing SSH with two factor authentication using Google Authenticator on a VM but it didn't succeed (I managed to login with the gcloud compute shell with no additional code).
[Jan 12th]
Some updates:
Google developer console works perfectly. Thanks.
For 2-step verification with the compute-engine SSH access, I retried everything all over again. Followed the instructions mentioned in the links provided, and did the following:
I created a new Google-Cloud project.
I used 2 different OS instances - Debian 8.2 and Ubuntu 15.10.
All of these tests failed - there was no prompt for a verification code.
I looked around in the Google compute-engine documentation, and they mention explicitly they support only certificate authentication (rather than username/password), so I cannot verify whether this is the root cause.
Is there anyone using 2-step verification with Google compute-engine?
Thanks
At last - a solution (thanks for Google cloud support).
A couple of updates on top of the document I have referred to:
Apart of adding a line to /etc/pam.d/sshd, one should also comment out the #include common-auth line. So it should be something like:
auth required pam_google_authenticator.so # from the original instructions
# #include common-auth # commenting out is new...
Apart of changing the ChallengeResponseAuthentication property in /etc/ssh/sshd_config, one should also add AuthenticationMethods publickey,keyboard-interactive in the following line:
ChallengeResponseAuthentication yes # from the original instructions
AuthenticationMethods publickey,keyboard-interactive # this is new...
Of course, this is on top of the regular instructions of installing libpam-google-authenticator, changing the sshd and sshd_config (as mentioned above), restarting the ssh/sshd service, and setting up the google-authenticator for the account.
Finally, a few more points:
Consider this carefully - from restarting the ssh/sshd account, no one can login without proper 2FA. So make sure anyone who should have ssh access - configured it properly.
I'm contemplating whether this is the proper solution for us, as it requires setup the VMs (each VM separately), and manual setting up the authenticator per each account and each VM manually. Not sure how scalable is this alternative. I would appreciate your thoughts...
Last but not least - the setup of libpam-google-authenticator may be simplified by using apt-get, no need for manually installing all dependencies and building it. Worked for me by running:
sudo apt-get install libpam-google-authenticator
Good Luck!

How to clone/copy your instance on Compute Engine?

I created a VM Instance with Ubuntu Precise. Then I installed tons of libraries and set up my environment. I will be using another server with slightly different configurations therefore I want to copy or clone my current instance.
I thought the clone button which appears when I edit an instance was going to do the thing I desired but after I cloned my instance and go into the server, I haven't see any library installed or whatever.
Is there any way to completely create a new copy of a server (instance) like I can in Amazon?
Create a snapshot of your current instance
Create a new instance with this snapshot as a disk
There you go.
There is no native way to clone using gcloud. However you can fake it.
Use the UI console for this.
Navigate to the compute you wish to clone, and click clone.
At the bottom of the clone screen, click the 'Equivalent Rest or command line'; command line for the gcloud commands.
Copy that over to your console and execute the gcloud command.

Is it possible to use OpenShift without using rhc?

I am trying to get an application running on OpenShift but after trying to create an ssh key on Ubuntu using ssh-keygen I ran into permissions problems. This is because I find I have no need for the rhc client if it only automates this process but bloats my computer (laptop) with a ruby installation.
I find that it would be best to have an alternative for Ubuntu (Linux) users. Is it possible to make this happen or do I have to go the rhc way?
You get a long way without the rhc command line tool. Obviously you can create your ssh key yourself and add/mange it via the OpenShift website. You can also create your application there and add cartridges. When it comes to starting the app, you can usually do that by jsut pushing your git repository. Last but not least, you can ssh onto your OpenShift gear and do a lot from there, for example view the log files.
That said, the rhc client is your one stop client for all this (and more). So even if you might not need it right now and some task are in fact done easier without it, I would still recommend to install it. A lot of information/tutorials are using rhc and w/o enough experience you will not know how to achieve a certain task in a different way.

Gnome 3 automatic execution of a script that needs network

my old father is using ubuntu-gnome. He has no static ip address. In order to perform remote administration, I need to know his ip. I was using dyndns free account (configuration in the adsl modem), but this will stop working in a couple of days.
I would like to run a script each time he logs in to publish his ip on my website. I have tried to put a script on the boot, but the network is not available. It seems that it is gnome 3 that starts the network, but I do not know much about gnome 3.
How should I do to have my script run automatically as soon as the network is available ?
One possible non-elegant solution for this is to put your script in his cron to run every X minutes :)
Looking to mine /etc/NetworkManager/ looks like there is a folder dispatcher.d that I think it'll do what you want. Just experiment with a bash/perl/python w/e script in there set the permission appropriately. You can find the UUID in the system-connections/ folder. More information is available in man networkmanager.
EDIT: Look what I found: https://askubuntu.com/questions/13963/call-script-after-connecting-to-a-wireless-network. Seems like this is exactly what you want.
The easiest way is to use another dynamic DNS service. I used to use my own. You could also put curl or wget command to cron or create a systemd service that will call that command periodically. As a target you would have to use your machine with a web server where you can see the IP in your logs.
It is not Gnome that connects the network, it is a system service called NetworkManager. It tries to connect at boot if possible. In some cases it waits for wireless signal, in other cases it waits for a user password. I recently verified that in Fedora, NetworkManager properly implements the systemd's network-online.target but it may have yet to be fixed in other distributions, see the upstream bug report.
https://bugzilla.gnome.org/show_bug.cgi?id=728965
If you want to run a system service just after boot, you need to use:
[Unit]
...
Wants=network-online.target
After=network-online.target
You could also just run a script that calls nm-online at the beginning to wait for the network connectivity if you can expect the connectivity to come up in reasonable time, otherwise it times out. Such a script can be run from any environment including a user session.
And, as noted already, you can put a script into /etc/NetworkManager/dispatcher.d that will be called on any network configuration change and such a script can then filter connection up events and start the notification script.