click to deploy Hadoop on GCE not working - google-compute-engine

I'm trying "Click-to-deploy Hadoop on Google Compute Engine" here
Unfortunately this doesn't seems to work : either the process stops almost immediately, or it's like it's frozen.
message displayed is
Deployment may take 3 to 10 minutes to complete, depending on the size of your cluster
Creating deployment
In any case, I can't have any cluster. Tried several zones, Hadoop versions, nothing.
Any thought ?

The problem is occurring because your Cloud project does not have a project id associated with it, but only a project number, which is true for some long-standing Cloud projects.
https://developers.google.com/console/help/new/#projectnumber
You can fix this by going into Developers Console, selecting your project from the project list, selecting Billing & settings from the left-hand navigation, and adding the project id there.
The following URL should take you there directly:
https://console.developers.google.com/project/_/settings
Thanks,
-Matt

A few items to help diagnose the problem:
Go to the Compute Engine instance list and check if there are any instances created for the deployment.
Check if there are any errors raised to the Javascript Console for your browser.
BTW, what browser and version are you using?
Thanks.

No instance deployed (however I can (and had) deployed compute engine VM instances)
I have a 404 in console :
POST https://console.developers.google.com/m/deploy?pid=1090158225078&cmd=custom…ion=europe-west1&app=hadoop&xsrf=R5Ezthkrr1L8xU1STye3sXUiHiA:1414055456964 404 (Not Found)
on Chrome, Windows7
I tried on Firefox too : no 404 in console but same effect : no deployment at all.

The "customdeploy" command should not be returning a 404, so let's check if there's something going on with your Cloud project.
Click to Deploy uses the preview version of Deployment Manager on the backend. Let's check the objects (if any) that Deployment Manager has created for the Hadoop deployment.
To do this, you will need to:
Install the Google Cloud SDK (if you have not already)
Add the preview component
Query for Deployment Manager templates
Query for Deployment Manager deployments
Install the Google Cloud SDK:
Instructions are here: https://cloud.google.com/sdk/
Add the preview component:
gcloud components update preview
Query for Deployment Manager templates
gcloud preview --project=<projectid> deployment-manager templates list
Query for Deployment Manager deployments
gcloud preview --project=<projectid> deployment-manager deployments --region europe-west1 list
One last question. Is this a relatively "new" or "old" Google Cloud project? Sometimes old projects need a feature to be enabled that is automatically enabled on new projects.
Thanks.

Related

Versioned deployment seems to be redirecting to HEAD deployment after OAuth login

I have a google workspace addon which is extending gmail and is using app script oauth to connect our own system, https://github.com/googleworkspace/apps-script-oauth2.
Our system has a development environment and a live environment, each uses a different identity client id secret which need to be used when connecting to each. I have stored these as different project properites with the development properties being prefixed with 'dev-' and i have a function that looks like this:
getScriptPropery function. This function has a variable that should let me change the script properties I access.
I have then created a versioned deployment that uses the live project properties and have kept my HEAD deployment using the develpoment properties. My problem is that once someone tries to login on the versioned deployment they seem to be redirected back to the HEAD version of the code which is using development project properties, this then breaks the login attempt and gives the following error: AppScript login error. I have tested this by adding logging that is only in the HEAD version of the getScriptProperty function and you can see the logs swap to using the development properties after redirecting from the login attempt, Logging example.
Is there a better way to manage deployments/project properties to stop this from happening or have a just missed something with how they are meant to work?

Legacy GCE and GKE metadata requests from google_daemon/manage_addresses.py

I have an old Debian Compute Engine instance (created and running since December 2013) and got an email warning about the turndown of Legacy GCE and GKE metadata server endpoints (more details at https://cloud.google.com/compute/docs/migrating-to-v1-metadata-server).
I followed the directions for locating the process and found that the requests were coming from /usr/share/google/google_daemon/manage_addresses.py. The script seems to be the same as what's at https://github.com/gtt116/gce/blob/master/google_daemon/manage_addresses.py (also with what's in that directory).
I don't recall installing this, so I'm imaging it came with the provided Debian image I used in 2013.
Does anyone know what this manage_addresses.py script is, what it does, and what I should do with it now that the legacy metadata server endpoints are turning down? Is it safe to just stop running it? Or is there a new script I should replace it with? Or should I just try to update it myself to use the new endpoint?
I dug around and was able to trace /usr/share/google/google_daemon/manage_addresses.py as being installed by a package called google-compute-daemon. A search for that brought me to https://github.com/GoogleCloudPlatform/compute-image-packages#troubleshooting which explains that google-compute-daemon has been replaced with python-google-compute-engine. That led me to https://cloud.google.com/compute/docs/images/install-guest-environment . I followed the instructions there and manually installed the guest environment.
I noticed during installation that it said it was removing the google-compute-daemon package (and a packaged called google-startup-scripts), so this seems like the right thing. And I'm no longer seeing any requests to the legacy endpoints. So it seems like at some point the old guest environment failed to update.
TLDR; If you have this problem, follow the instructions at https://cloud.google.com/compute/docs/images/install-guest-environment#installing_guest_environment to manually update the guest environment.

Urbancode IBM getting Versions

Is there any Rest API that can be used to get the versions of an application in DEV Environment in Urbancode.
I tried using supported Rest API to get List of application and Snapshots(If any) but now I need to get the versions (if any) of that particular application inside DEV Environment.
How do I go about it ?
Any suggestions would be appreciated.
Thanks.
In IBM UrbanCode Deploy, applications don't have versions. Components have versions.
If you've got the snapshot that has been deployed to the environment, you can get the component versions in that snapshot with this command:
https://www.ibm.com/support/knowledgecenter/SS4GSP_6.2.4/com.ibm.udeploy.api.doc/topics/rest_cli_snapshot_getsnapshotversions_get.html
If you don't have a snapshot, you can use the internal, unsupported API to get the current desired inventory. The command would be something like this:
GET https://ucdserver.example.com:8443/rest/deploy/environment/9e022848-ca4f-447e-9311-3d77103c612c/latestDesiredInventory/true?rowsPerPage=50&pageNumber=1&orderField=name&sortType=desc
That command returns JSON with the versions that have most recently been deployed to the environment and a lot of other info about the environment inventory.

Run IBM MobileFirst Platform Foundation on IBM Containers

I am following the MobileFirst containers tutorial:
https://mobilefirstplatform.ibmcloud.com/tutorials/en/foundation/7.1/ibm-containers/run/
I have already created mobilefirst project. Now I want to use that project war file. As per above link, all installation process is done. Now I am facing an issue while running the scripts. When I try to run prepareserverdbs.sh it asks me to specify the name of my Bluemix application (mandatory). Which name should I provide there?
I tried with same war file name which I created but its throwing The Bluemix application Test was not found. I have already copied war file in usr/projects folder.
I guess you skipped step 3: Using the configuration files. If you choose to run the scripts interactively, you can skip the configuration but it is strongly suggested to at least read and understand the arguments you will need to provide (in the prepareserverdbs.properties collapse you can find instructions how to create a database service).
In order to create a database service and bind it to a Bluemix application, go to the Bluemix Dashboard, create a new Cloud Foundry application and follow the instructions:
Choose "web".
Choose the Liberty for Java option.
Name your app.
Note: Choose a unique name.
Click on the "view app overview" button.
In the overview page click on the "add a service or API" button.
Choose a DB service type (sqldb, cloudantNoSQLDB) from the Data and Analytics services section.
Name the service and click on the "create" button

VM Instance is not accessible The project you requested is unavailable

Just signed to the free Google cloud account (300$ Credit) to see if it supports exporting VMs in OVF format.
Created a new project and By clicking on the Compute>Compute engine> VM Instance I see below error message:
"The project you requested is unavailable."
There is no extra information provided on the screen.
Google compute engine currently doesn't support exporting VMs in OVF or OVA format. You can use free tools, e.g. VirtulBox to convert GCE images from the RAW format to VMD, VDI, VHD
https://www.virtualbox.org/manual/ch08.html#idp58756992
Here are instructions of how to create a RAW file from an image:
https://cloud.google.com/compute/docs/images#exporting_an_image_to_google_cloud_storage
In order to troubleshoot your GCE "The project you requested is unavailable" issue in developer console, you can login using Incognito window in chrome or use another browser to see if that resolves the issue.
Update: GCE supports OVA and OVF now. Here is how you can import the image.