I would like to use jupyter notebook from google cloud compute engine. When i try to launch it through command line, I am not able to open the notebook using my browser.
Please let me know on how to do this.
It looks like you're trying to start a Jupyter notebook server on the VM and want to access it using the external IP of the VM (assuming you have not disabled the external IP option on your VM).
You will need to do the following:
Modify jupyter_notebook_config.py in your ~/.jupyter directory. Do go through exactly what you need to modify and how to secure your notebook server since Jupyter notebook by default only listens on the loopback interface (i.e. 127.0.0.1 aka localhost).
The minimum set of configuration options that you should uncomment and
edit in jupyter_notebook_config.py is the following:
# Set options for certfile, ip, password, and toggle off
# browser auto-opening
c.NotebookApp.certfile = u'/absolute/path/to/your/certificate/mycert.pem'
c.NotebookApp.keyfile = u'/absolute/path/to/your/certificate/mykey.key'
# Set ip to '*' to bind on all interfaces (ips) for the public server
c.NotebookApp.ip = '*'
c.NotebookApp.password = u'sha1:bcd259ccf...<your hashed password here>'
c.NotebookApp.open_browser = False
# It is a good idea to set a known, fixed port for server access
c.NotebookApp.port = 9999
You will need to modify the firewall rules to allow ingress traffic to the port (on the VM) you just configured in the previous step. To do that I will recommend tag based firewall rules so that you can control which VMs the firewall rule applies to.
Network tags are used by networks to identify which instances are
subject to certain firewall rules and network routes. For example, if
you have several VM instances that are serving a large website, tag
these instances with a shared word or term and then use that tag to
apply a firewall rule that allows HTTP access to those instances. Tags
are also reflected in the metadata server, so you can use them for
applications running on your instances. When you create a firewall
rule, you can provide either sourceRanges or sourceTags but not both.
# Assuming Jupyter notebook is running on port 9999
# Add a new tag based firewall rule to allow ingress tcp:9999
gcloud compute firewall-rules create rule-allow-tcp-9999 --source-ranges 0.0.0.0/0 --target-tags allow-tcp-9999 --allow tcp:9999
# Add the allow-tcp-9999 target tag to the VM named say 'vm-1'
gcloud compute instances add-tags vm-1 --tags allow-tcp-9999
# If you want to list all the GCE firewall rules
gcloud compute firewall-rules list
It might take a few seconds to couple of minutes for the changes to take effect.
Alternatively, you can also use Google Cloud Console instead of gcloud to configure firewall rules. You can go through this answer which explains that in detail.
You can also "Create Firewall Rules" to allow your jupyter c.NotebookApp.port number.
refer this image set Protocols and ports to tcp:<jupyter port number>
Related
I created a simple f1 micro instance (Ubuntu 16.10) on Google Compute Engine. I had created a VPC (Networking) and allowed http/https. I explicitly created a Firewall rule to allow SSH (tcp:22). But I am not able to login to the VM instance.
I keep getting this error:
Here is my Firewall Rule:
How to fix this?
I was able to finally ssh into an instance by Choosing "Automatic" while creating a new VPC network instead of Custom and also chose allow ssh. This added default Firewall rules which were more accurate.
For anyone that runs into this problem I reached out to Google support by email (who responded!!) and they instructed me to run the following commands from terminal (where I have gcloud installed):
If you haven't yet authenticated / or to set the correct project
gcloud auth login
gcloud config set project PROJECT_ID
This is the meat of it - setting the firewall rules.
gcloud beta compute firewall-rules create default-allow-ssh --allow tcp:22 --priority=65534
Also - use chrome - the interface seems to occasionally throw a fit whenever you use any other browser.
I'm trying to have my GCE instance listen on multiple IP addresses (for SEO reasons - to host multiple low traffic sites on the same instance).
Final objective: mydomain.com points to IP1, myotherdomain.es points to IP2, the GCE instance will listen on both IP1 and IP2 and serve content accordingly.
I added a target instance pointing to my main instance and managed to create a forwarding rule like this:
gcloud compute forwarding-rules create another-ip --port 80 --target-instance MY_TARGET_INSTANCE_URL
It actually created an ephemeral IP address; I tried to promote it to static but I exceeded my quota (I'm currently on my 2 months free trial).
Is this correct though? Will I be able to create any number of static IPs and point them to my only instance once the trial ends? I also couldn't find anything about pricing: I know an IP assigned to an active instance is free, but what about additional ones?
Since this is a necessary configuration for a site I'm managing, I'd like to be sure it works before committing to moving everything on GCE.
You can get multiple external IPs for one VM instance with forwarding rules.
By default, VM will be assigned with an ephemeral external IP, you can promote it to static external IP, which will remain unchanged after stop and restart.
Extra external IPs have to be attached to forwarding rules which point to the VM. You can use (or promote to) static IPs as well.
The command you may want to use:
Create a TargetInstance for your VM instance:
gcloud compute target-instances create <target-instance-name> --instance <instance-name> --zone=<zone>
Create a ForwardingRule pointing to the TargetInstance:
gcloud compute forwarding-rules create <forwarding-rule-name> --target-instance=<target-instance-name> --ip-protocol=TCP --ports=<ports>
See Protocol Forwarding.
I am also need 2 static ips for one compute engine instance but google's quota is not allow this.
You can see your quotas from https://console.cloud.google.com/iam-admin/quotas
An other possibility is to have multiple network interface on the VM
This require adding a new VPC network, the ip 10.130.0.0/20 is not used on the current infrastructure and can be used as an additional network, you would add the proper firewall rules and the proper routing rules (you can copy the default one to avoid any miss-configuration)
Note that you can not add a network interface to an existing machine, you would need to
Turn off the current machine
Detach disk and network (without deleting them !!!)
Create a new machine with 2 network cards or more
Attach the old disk and network to the new machine
Finally you would need to pay attention to the default gateway, the classic network behavior would make everything go through the first network interface the second won't be accessible until you change the default gateway and or create the proper routing rules.
Typically you have eth0 and eth1 this example makes eth1 available to services that bind to eth1
ip addr add 10.130.0.2/32 broadcast 10.130.0.2 dev eth1
ip link set eth1 up
ip route add 10.130.0.1 src 10.130.0.2 dev eth1
ip route add 10.130.0.1 src 10.130.0.2 dev eth1 table 100
ip route add default via 10.130.0.1 dev eth1 metric 10
ip route add default via 10.130.0.1 dev eth1 table 100
ip rule add from 10.130.0.2/32 table 100
ip rule add to 10.130.0.2/32 table 100
curl --interface eth1 ifconfig.co
curl --interface eth0 ifconfig.co
ping -I eth1 8.8.8.8
Here is the documentation, alternatively this guide may help.
Is there a network-level IP address blocking/blacklist capability to a Google Compute Engine instance? For example, a site is hosted on a GCE instance to allow public users access. However, a malicious script runs several times/second which is not legitimate traffic. Ideally, the IP of the offending user could be placed on a block list so traffic would not be routed to the instance, rather than just server side only mechanism (apache modules, IPtables, etc) which still requires CPU/RAM/disk resources.
You can setup an HTTP load balancer for your instances, and allow traffic only from the LB IP address to your instances. More information can be found in this Help Center article.
GCP does not provide WAF natively. You can use marketplace WAF (like Brocade WAF) to block IPs.
https://cloud.google.com/launcher/solution/brocade-public-1063/stm-csub-1000-h-saf?q=brocade
This is absolutely not the recommended way to manage your firewall blacklist.
However...
In the compute GUI, you can create a firewall rule set action on match to "deny" and protocols and port to "deny all". Then set source IPs.
compute GUI
You could then run a cron job to update your firewall through gcloud compute firewall-rules update to update source IPs should your list change.
Note (from Google - https://cloud.google.com/vpc/docs/using-firewalls):
gcloud compute firewall-rules update is used to update firewall rules that allow/deny incoming/outgoing traffic. The firewall rule will only be updated for arguments that are specifically passed. Other attributes will remain unaffected. The action flag (whether to allow or deny matching traffic) cannot be defined when updating a firewall rule
Yes you can block it using Gcloud Firewall.
Try creating the firewall rule from the command line or by logging into Google Cloud.
Example:
gcloud compute firewall-rules create tcp-deny --network example-network --source-ranges 10.0.0.0/8 --allow !tcp:80
Above Rule will block the range 10.0.0.0/8 to port 80 (tcp).
Same can be done to block other IP Ranges over tcp and udp.
For more info check this: glcoud network config
I am a very basic user in Google Cloud Platform.
Is it possible to use a GUI of my VM instance ? I am currently using Centos7 VM.
You can use VNC to connect to VMs on Google Compute Engine. Here's a detailed tutorial for how to set this up.
For added security:
use a long, complex password (though note that VNC limits passwords to 8 characters)
instead of opening up port 5901 to the Internet, consider using an SSH tunnel. This is more complex, and depending on your Internet connection, may slow down your graphics refresh rate, but will be more secure.
To use the alternative approach with an SSH tunnel, here are the differences from the tutorial you need to follow:
don't open port 5901 in the Google Compute Engine firewall
create an SSH tunnel from your desktop/laptop to GCE VM via:
gcloud compute ssh \
${VM_INSTANCE} \
--project $PROJECT \
--zone $ZONE \
--ssh-arg "-L ${LOCAL_PORT}:localhost:5901"
where you need to provide the right parameters for ${VM_INSTANCE}, $PROJECT, and $ZONE that match your configuration. You can choose ${LOCAL_PORT} to be 5901 if you wish, but if you decide to VNC into several different GCE VM instances, you'll have to choose unique ports for your local machine.
You need to keep this connection open to use VNC. If this connection is closed, you will lose VNC access as well.
Instead of connecting to your VM using its external IP, connect via localhost:${LOCAL_PORT} with ${LOCAL_PORT} same as selected earlier in step #2
My need was to connect a Windows TightVNC client to Google Compute Engine Cloud Instance of Debian 10 (Buster). The various tutorials I have worked through omitted one important step: make sure the vnc server is not restricted to localhost.
The essential steps for Google Cloud are summarized as
confirm you have a running VM instance and that you have ssh access.
I explicitly disabled enable-oslogin (how to disable oslogin)
and loaded my own Puttygen-created SSH certificate.
in VPC Networks > Network Interface Details > Firewall and routes > Rules add a rule to allow ingress for ip range 0.0.0.0/0 (or a
known limited range), for tcp:5900-5920 (this allows for up to 20
VNC instances)
set up the VNC server (tutorials here and for debian 9
here and for debian 10 here and more complete and recent
here for debian 10
after doing this, I could not get past "Connection refused."
Missing step: make sure -localhost no is included as argument when starting the vncserver:
vncserver -localhost no
Once all these conditions were satisfied, I had desktop access.
I have deployed a hadoop cluster on google compute engine. I then run a machine learning algorithm (Cloudera's Oryx) on the master node of the hadoop cluster. The output of this algorithm is accessed via an HTTP REST API. Thus I need to access the output either by a web browser, or via REST commands. However, I cannot resolve the address for the output of the master node which takes the form http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091.
I have allowed http traffic and allowed access to ports 80 and 8091 on the network. But I cannot resolve the address given. Note this http address is NOT the IP address of the master node instance.
I have followed along with examples for accessing IP addresses of compute instances. However, I cannot find examples of accessing a single node of a hadoop cluster on GCE, that follows this form http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091. Any help would be appreciated. Thank you.
The reason you're seeing this is that the "HOSTNAME.c.PROJECT.internal" name is only resolvable from within the GCE network of that same instance itself; these domain names are not globally visible. So, if you were to SSH into your master node first, and then try to curl http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091 then you should successfully retrieve the contents, whereas trying to access from your personal browser will just fail to resolve that hostname into any IP address.
So unfortunately, the quickest way for you to retrieve those contents is indeed to use the external IP address of your GCE instance. If you've already opened port 8091 on the network, simply use gcutil getinstance CLUSTER_NAME-m and look for the entry specifying external IP address; then plug that in as your URL: http://[external ip address]:8091.
If you turned up the cluster using bdutil, a more involved but nicer way to access your cluster is by running the bdutil socksproxy command. This opens a dynamic-port-forwarding SSH tunnel to your master node as a SOCKS5 proxy, so that you can then configure your browser to use localhost:1080 as your proxy server, make sure to enable remote DNS resolution, and then visit your browser using the normal http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091 URL.