I want to deploy multiple cloud functions. Here is my index.js:
const { batchMultipleMessage } = require('./gcf-1');
const { batchMultipleMessage2 } = require('./gcf-2');
module.exports = {
batchMultipleMessage,
batchMultipleMessage2
};
How can I use gcloud beta functions deploy xxx to deploy these two functions at one time.
Option 1:
For now, I write a deploy.sh to deploy these two cloud functions at one time.
TOPIC=batch-multiple-messages
FUNCTION_NAME_1=batchMultipleMessage
FUNCTION_NAME_2=batchMultipleMessage2
echo "start to deploy cloud functions\n"
gcloud beta functions deploy ${FUNCTION_NAME_1} --trigger-resource ${TOPIC} --trigger-event google.pubsub.topic.publish
gcloud beta functions deploy ${FUNCTION_NAME_2} --trigger-resource ${TOPIC} --trigger-event google.pubsub.topic.publish
It works, but if gcloud command line support deploy multiple cloud functions, that will be best way.
Option 2:
https://serverless.com/
If anyone is looking for a better/cleaner/parallel solution, this is what I do:
# deploy.sh
# store deployment command into a string with character % where function name should be
deploy="gcloud functions deploy % --trigger-http"
# find all functions in index.js (looking at exports.<function_name>) using sed
# then pipe the function names to xargs
# then instruct that % should be replaced by each function name
# then open 20 processes where each one runs one deployment command
sed -n 's/exports\.\([a-zA-Z0-9\-_#]*\).*/\1/p' index.js | xargs -I % -P 20 sh -c "$deploy;"
You can also change the number of processes passed on the -P flag. I chose 20 arbitrarily.
This was super easy and saves a lot of time. Hopefully it will help someone!
Related
When I try to use a gcloud CLI to deploy a small python script that listens to Firestore events, the script fails to listen to the Firestore events. If I use the web inline UI or web zip upload, the script actually listens to Firestore events. The command line doesn't show any errors.
Deploy script
gcloud beta functions deploy print_name \
--runtime python37 \
--service-account <myprojectid>#appspot.gserviceaccount.com \
--verbosity debug \
--trigger-event providers/cloud.firestore/eventTypes/document.create \
--trigger-resource projects/<myprojectid>/databases/default/documents/Test/{account}
main.py
def print_name(event, context):
value = event["value"]["fields"]["name"]["stringValue"]
print("New name: " + str(value))
gcloud --version
Google Cloud SDK 243.0.0
beta 2019.02.22
bq 2.0.43
core 2019.04.19
gsutil 4.38
Back to comments
The document is pretty basic (has a name string field).
Any ideas? I'm curious if the gcloud CLI has a bug.
The inline web UI and zip uploader work great. I've tried multiple variations of this (e.g. removing 'beta', adding and removing different deploy args).
I'd expect the script to actually listen to Firestore events.
The "default" in trigger-resource needs parentheses around it.
gcloud beta functions deploy print_name \
--runtime python37 \
--service-account <myprojectid>#appspot.gserviceaccount.com \
--verbosity debug \
--trigger-event providers/cloud.firestore/eventTypes/document.create \
--trigger-resource "projects/<myprojectid>/databases/(default)/documents/Test/{account}"
I have a bunch of GCE instances and I want to run the same shell command on all of them. Is it possible to do something like gcloud compute ssh --command="ls -al" my-instance1 my-instance2 my-instance3?
You can use gcloud compute instances list --format='value[separator=","](name,zone)' to get a list like:
my-instance1,my-zone1
my-instance2,my-zone2
my-instance3,my-zone3
Then you can use bash Substring Removal to extract the parts before and after the comma.
var="before,after"
before="${var%,*}"
after="${var#*,}"
Put it all in a loop and add trailing '&' to run things in the background:
for instance in $(gcloud compute instances list --format='value[separator=","](name,zone)'); do
name="${instance%,*}";
zone="${instance#*,}";
gcloud compute ssh $name --zone=$zone --command="ls -al" &
done
To add to Mike's answer, here is how to do the same without Substring Removal:
for i z in $(gcloud compute instances list --format='value(name, zone)'); do
gcloud compute ssh $i --command="ls -lah" --zone=$z;
done
As per the documentation at https://cloud.google.com/sdk/gcloud/reference/init gcloud init myproject command does not work.
google-cloud> gcloud init myproject
Initialized gcloud directory in [/Users/arungupta/workspaces/google-cloud/myproject/.gcloud].
Cloning [https://source.developers.google.com/p/myproject/r/default] into [default].
Cloning into '/Users/arungupta/workspaces/google-cloud/myproject/default'...
fatal: remote error: Repository not found.
You may need to create a repository for this project using the Source Code tab at https://console.developers.google.com
ERROR: Command '['git', 'clone', 'https://source.developers.google.com/p/myproject/r/default', '/Users/arungupta/workspaces/google-cloud/myproject/default', '--config', 'credential.helper=gcloud.sh']' returned non-zero exit status 128
ERROR: Unable to initialize project [myproject], cleaning up [/Users/arungupta/workspaces/google-cloud/myproject].
ERROR: (gcloud.init) Unable to initialize project [myproject].
Creating a project using gcloud init minecraft-server --project minecraft-server-183 creates the project with the name minecraft-server-183.
The project so created is then not visible at https://console.developers.google.com/project.
What is the correct gcloud command to create a new project, without going to the console?
It is now possible with the gcloud alpha projects create command.
For more information see: https://cloud.google.com/resource-manager/
Just wanted to complete the circle here.
Google Cloud CLI tool 'gcloud' supports creating of projects without the need for the 'alpha' component installed from the version 147.0.0 (March 15, 2017) onwards.
Official Reference Link: https://cloud.google.com/sdk/gcloud/reference/projects/create
Release Notes for v147.0.0:
https://cloud.google.com/sdk/docs/release-notes#14700_2017-03-15
It is mentioned under subheading of Google Cloud Resource Manager
For quick reference
Synopsis
gcloud projects create [PROJECT_ID] [--no-enable-cloud-apis] [--folder=FOLDER_ID] [--labels=[KEY=VALUE,…]] [--name=NAME] [--organization=ORGANIZATION_ID] [--set-as-default] [GCLOUD_WIDE_FLAG …]
Description
Creates a new project with the given project ID. By default, projects are not created under a parent resource. To do so, use either the --organization or --folder flag.
Sample Code
gcloud projects create example-foo-bar-1 --name="Happy project" --labels=type=happy
Here's a script that will create a project that is editable by a user (for many reasons, such as for auditability of service accounts, you might want to create per-user projects):
#!/bin/bash
if [ "$#" -lt 3 ]; then
echo "Usage: ./create_projects.sh billingid project-prefix email1 [email2 [email3 ...]]]"
echo " eg: ./create_projects.sh 0X0X0X-0X0X0X-0X0X0X learnml-20170106 somebody#gmail.com someother#gmail.com"
exit
fi
ACCOUNT_ID=$1
shift
PROJECT_PREFIX=$1
shift
EMAILS=$#
gcloud components update
gcloud components install alpha
for EMAIL in $EMAILS; do
PROJECT_ID=$(echo "${PROJECT_PREFIX}-${EMAIL}" | sed 's/#/-/g' | sed 's/\./-/g' | cut -c 1-30)
echo "Creating project $PROJECT_ID for $EMAIL ... "
# Create project
gcloud alpha projects create $PROJECT_ID
# Add user to project
gcloud alpha projects get-iam-policy $PROJECT_ID --format=json > iam.json.orig
cat iam.json.orig | sed s'/"bindings": \[/"bindings": \[ \{"members": \["user:'$EMAIL'"\],"role": "roles\/editor"\},/g' > iam.json.new
gcloud alpha projects set-iam-policy $PROJECT_ID iam.json.new
# Set billing id of project
gcloud alpha billing accounts projects link $PROJECT_ID --account-id=$ACCOUNT_ID
done
Explanation of the script is on medium: https://medium.com/google-cloud/how-to-automate-project-creation-using-gcloud-4e71d9a70047#.t58mss3co and a github link to the above code (I'll update it to remove the alpha when it goes beta/GA, for example) is here: https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/blogs/gcloudprojects/create_projects.sh
Update: as of 10/24/2016 #poolie says the gcloud command mentioned in Stephen's answer is now publicly accessable, will leave this answer here as I give some other usage suggestions.
I also have the problem, and was extremely discouraged by #Stephan Weinberg's remark, but I noticed when doing gcloud init that it asks where to put a "default" repository. so I looked at that one's config and see that it's slightly different from what's documented.
try pushing to https://source.developers.google.com/p/YOUR-PROJECT-NAME/r/default instead, it worked for me!
I am attempting to use drip to reduce JRuby VM startup time for a Rails app, per the JRuby wiki.
I see some speed up by changing some of the JRUBY_OPTS but I am not seeing any reduction using drip. It seems to be spinning up a new process/vm for each call instead of re-using the existing VM.
I was following this guide to get things setup.
dripmain.rb
require_relative './config/application'
terminal
$ { time ( jruby -e 'puts 9*9') }
81
( jruby -e 'puts 9*9'; ) 0.08s user 0.08s system 14% cpu 1.144 total
$ drip ps
31166 org.flatland.drip.Main org.jruby.Main ...
$ { time ( jruby -e 'puts 9*9') }
81
( jruby -e 'puts 9*9'; ) 0.08s user 0.08s system 13% cpu 1.182 total
$ drip ps
30965 org.flatland.drip.Main org.jruby.Main
I expected those process ids for drip to be the same. Open to hearing non-drip solutions as well. I looked into theine / nailgun but couldn't get them working either.
The problem was that I was missing this export:
export DRIP_INIT="" # Needs to be non-null for drip to use it at all!
Solution found as part of rvm shell script on github.
# Source this file if you use Drip to make the JVM not suck
# drip: https://github.com/flatland/drip
# https://github.com/flatland/drip/wiki/JRuby
# JAVACMD is honored by jruby
export JAVACMD=`which drip`
# Drip preloads our codez
export DRIP_INIT_CLASS=org.jruby.main.DripMain
export DRIP_INIT="" # Needs to be non-null for drip to use it at all!
# settings from: https://github.com/jruby/jruby/wiki/Improving-startup-time
export JRUBY_OPTS="-J-XX:+TieredCompilation -J-XX:TieredStopAtLevel=1 -J-noverify -X-C"
I have a situation where only root can mailx, and only ops can restart the process. I want to make an automated script that both restarts the process and sends an email about doing so.
When I try this using a function the function is "not found".
I had something like:
#!/usr/bin/bash
function restartprocess {
/usr/bin/processcontrol.sh start
}
export -f restartprocess
su - ops -c "restartprocess"
mailx -s "process restarted" myemail.mydomain.com < emailmessage.txt
exit 0
It told me that the function was not found. After some troubleshooting, it turned out that the ops user's default shell is ksh.
I tried changing the script to run in ksh, and changing "export -f" to "typeset -xf", and still the function was not found. Like:
ksh: exportfunction not found
I finally gave up and just called the script (that was in the function directly) and that worked. It was like:
su - ops -c "/usr/bin/processcontrol.sh start"
(This is all of course a simplification of the real script).
Given that user ops has default shell is ksh and I can't change that or modify sudoers, is there a way to export a function such that I can su as ops (and I need to run ops's profile) and execute that function?
I made sure ops user had permission to the directory of the script I wanted it to execute, and permission to run that script.
Any education about this would be appreciated!
There are many restrictions for exporting functions, especially
combined with su - ... with different accounts and different shells.
Instead, turn your script inside out and put all of the command
that is to be run inside a function in the calling shell.
Something like: (Both bash and ksh)
#!/usr/bin/bash
function restartprocess {
/bin/su - ops -c "/usr/bin/processcontrol.sh start"
}
if restartprocess; then
mailx -s "process restarted" \
myemail#mydomain.com < emailmessage.txt
fi
exit 0
This will hide all of the /bin/su processing inside the restartprocess function, and can be expanded at will.