I have the following lines of python code
import os
def hello_world():
r=os.system("curl ipinfo.io/ip")
print (r)
hello_world()
Shows the desired output when executed from command line in Google Cloud Shell but seems there is a 0 at the end of IP Address output
$ python3 main2.py
34.X.X.2490
When I deployed the same code in Google CLoud function it is showing OK as output
I have to replace the first line of code in GCF as follows to make it deploy.
def hello_world(self):
Any suggestion so that GCF displays the desired output which is the output of curl command?
Your function won't work for 2 reasons:
Firstly, you don't respect the HTTP Cloud Function Python function signature:
def hello_world(request):
....
Secondly, you can't use system call. In fact not exactly, you can perform system call, but, because you don't know which package/binaries are installed, you can't rely on this. It's serverless, you don't manage the underlying infrastructure and runtime environment.
Here you made the assumption that CURL is installed on the runtime image. Maybe yes, maybe not, maybe it was, maybe it will be remove in future!! You can't rely on that!!
If you want to manage you runtime environment, you can use Cloud Run. You will manage your runtime environment, and you can install what you want on it and then you are sure of what you can do.
Last remarks:
note: instead of performing a CURL, you can perform a http get request to the same URL to get the IP
Why do you want to know the outgoing IP? It's serverless, you also don't manage the network. You will reach the internet through a Google IPs. It can change everytime, and other cloud functions (or cloud run), from your projects or project from others (like me), are able to use the same IPs. It's Google IPs, not yours! If it's your requirement, let me know, there are solutions for that!
Related
we have a Google cloudbuild pipeline that does the following:
Create a temp function
Run some tests
Deploy the production function
However, the second step often fails due to the first container not being ready yet:
Step #3: Unexpected token '<' at 2:1
Step #3: <html><head>
Step #3: site unreachable
looks like it is returning some placeholder html from nginx.
How can we fix that?
Currently, we just put an ugly sleep between steps
You probably want to have a look to the Cloud Functions API, there you can find the operations endpoint that will tell you if the operation is finished or not (assuming v1, otherwise look below): https://cloud.google.com/functions/docs/reference/rest/v1/operations/get
The operation is the same Id that is returned in the creation operation. Also you can list them with the list endpoint (in the same doc).
I'm using pulumi azure native for infrastructure as code. I need to create an Azure Web App (based on an App Service Plan) and add some app settings (and connection strings) throughout the code, e.g., Application Insights instrumentation key, Blob Storage account name, etc.
I figured out that there is a method, WebAppApplicationSettings, that can update web app settings:
from pulumi_azure_native import web
web_app = web.WebApp(
'my-web-app-test123',
...
)
web.WebAppApplicationSettings(
'myappsetting',
name=web_app.name,
resource_group='my-resource-group',
properties={'mySetting': 123456},
opts=ResourceOptions(depends_on=[web_app])
)
It turns out that WebAppApplicationSettings replaces the entire app settings with the value given in the properties parameter, which is not what I need. I need to append a new setting to the existing settings.
So, I tried this:
Fetch the existing settings from web app using list_web_app_application_settings_output
Add the new settings the existing settings
Update the app settings using WebAppApplicationSettings
from pulumi_azure_native import web
app = web.WebApp(
'my-web-app-test123',
...
)
current_apps_settings = web.list_web_app_application_settings_output(
name=web_app.name,
resource_group_name='my-resource-group',
opts=ResourceOptions(depends_on=[web_app])
).properties
my_new_setting = {'mySetting': 123456}
new_app_settings = Output.all(current=current_apps_settings).apply(
lambda args: my_new_setting.update(args['current'])
)
web.WebAppApplicationSettings(
'myappsetting',
name=app.name,
resource_group='my-resource-group',
properties=new_app_settings,
opts=ResourceOptions(depends_on=[web_app])
)
However, this doesn't work either and throws the following error during pulumi up:
Exception: invoke of azure-native:web:listWebAppApplicationSettings failed: invocation of azure-native:web:listWebAppApplicationSettings returned an error: request failed /subscriptions/--------------/reso
urceGroups/pulumi-temp2/providers/Microsoft.Web/sites/my-web-app-test123/config/appsettings/list: autorest/azure: Service returned an error. Status=404 Code="ResourceNotFound" Message="The Resource 'Microsoft.Web/sites/my-web-app-test123' under resource group 'pulumi-temp2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
error: an unhandled error occurred: Program exited with non-zero exit code: 1
Is there way that I can add a new app setting to Azure Web App using pulumi without changing/removing the existing settings?
Here's a suboptimal workaround: App Configuration and Enable Azure Function Dynamic Configuration.
And as far as I can tell it comes with some drawbacks:
cold start time may increase
additional costs
care must be taken to avoid redundant calls (costly)
additional boilerplate code needed for every function app
Maybe there's a better way, I mean I hope there is, I just haven't found it yet either.
After some searching and reaching out to pulumi-azure-native people, I found an answer:
Azure REST API doesn't currently support this feature, i.e., updating a single Web App setting apart from the others. So, there isn't such a feature in pulumi-azure-native as well.
As a workaround, I stored (kept) all the app settings I needed to be added, updated, or removed in a dictionary throughout my Python script, and then I passed them to the web.WebAppApplicationSettings class at the end of the script so that they will be applied all at once to the Web App resource. This is how I solved my problem.
I would like to serve a Next.js app in europe using Firebase Hosting & Functions capabilities.
I do understand from the doc that:
If you are using HTTP functions to serve dynamic content for Firebase
Hosting, you must use us-central1
and that
Firebase Hosting supports Cloud Functions in us-central1 only
It's pretty clear: you must use us-central. But my main target is Europe..
I've read the following on the Cloud Functions locations guide:
For HTTP and callable functions, we recommend that you first set your
function to the destination region, or closest to where most expected
customers are located, and then alter your original function to
redirect its HTTP request to the new function (they can have the same
name). [Solution 1] If clients of your HTTP function support
redirects, you can simply change your original function to return an
HTTP redirect status (301) along with the URL of your new function.
[Solution 2] If your clients do not handle redirects well, you can
proxy the request from the original function to the new function by
initiating a new request from the original function to the new
function. The final step is to ensure that all clients are calling the
new function.
I've highlighted what seems to be two solutions to my initial problem:
Solution 1
Have a us-central1 function that send a 301 redirection to https://europe-west1-[myProject].cloudfunctions.net/[myEuropeanFunction]
Have a europe-west1 function that does the job (in my case, serve the Next.js app)
Happily using Firestore located in europe-west1
This would only work if clients of the HTTP function support redirects. In my case, it's fine: all browsers support redirection.
exports.nextServer = functions
.https
.onRequest((req, res) => {
res.set('location', 'https://europe-west1-<my-project>.cloudfunctions.net/nextServerEurope');
res.status(301).send()
});
exports.nextServerEurope = functions
.region('europe-west1')
.https
.onRequest((req, res) => {
return server.prepare().then(() => nextjsHandle(req, res));
});
The issue with that solution is that the URL changes in the browser to https://europe-west1-.cloudfunctions.net/nextServerEurope :-/
Solution 2
Have a us-central1 function that initiate a new/proxy request to the europe-west1 function
Have the same europe-west1 function that does the job (in my case, serve the Next.js app)
Still happily using Firestore located in europe-west1
By proxy request (as suggested in the guide), it would mean using a lib like axios I suppose. I know there are some libraries to perform proxy request available for node as well.
However, with that solution, the first issue I can think of is the unnecessary delay introcuded by passing by the us endpoint:
client -> us endpoint -> eu endpoint -> do stuff -> us endpoint -> client
Billing wise, I'm wondering what would be the impact..
I know that two services from different regions calling each others can increase the latency and the billing (egress).
With the first solution, there's no egress traffic as it's only a redirection to the european endpoint. But the redirection itself is not a valid solution in my case.
It's unclear for me what would be the additional billing cost with the second solution (beside the latency cost): is the traffic for the proxy request from us to eu going to be expensive?
To wrap-up:
The solution 1 is easy but leads to a non-transparent redirection
The solution 2 seems ok but it requires extra http request which leads to extra-latency (and potentially extra billing)
In the end, both solutions doesn't seem quite okay.
Therefore my question:
How do you serve in Europe dynamic content using Firebase Hosting and Functions?
Firebase Hosting only supports Cloud Functions in Us-Central as you mentioned and as stated in the Firebase Hosting Official Documentation.
I have created a Feature Request in Public Issue Tracker to support other regions when using Firebase Hosting with Cloud Functions. Please note, there is no ETA when this will be implemented.
So as #Doug Stevenson suggest, you can use Firebase Hosting with Cloud Run instead to serve your Dynamic Content.
Just to update. As of August 2022.
Finally, latency issue can be solved easily for now.
Firebase Hosting rewrites to CF3 are able to be done to any CF3
region, not just us-central1.
Reference: Feature Request Ticket
I've been using the Google apiclient library in python for various Google Cloud APIs - mostly for Google Compute - with great success.
I want to start using the library to create and control the Google Logging mechanism offered by the Google Cloud Platform.
However, this is a beta version, and I can't find any real documentation or example on how to use the logging API.
All I was able to find are high-level descriptions such as:
https://developers.google.com/apis-explorer/#p/logging/v1beta3/
Can anyone provide a simple example on how to use apiclient for logging purposes?
for example creating a new log entry...
Thanks for the help
Shahar
I found this page:
https://developers.google.com/api-client-library/python/guide/logging
Which states you can do the following to set the log level:
import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
However it doesn't seem to have any impact on the output which is always INFO for me.
I also tried setting httplib2 to debuglevel 4:
import httplib2
httplib2.debuglevel = 4
Yet I don't see any HTTP headers in the log :/
I know this question is old, but it is getting some attention, so I guess it might be worth answering to it, in case someone else comes here.
Stackdriver Logging Client Libraries for Google Cloud Platform are not in beta anymore, as they hit General Availability some time ago. The link I shared contains the most relevant documentation for installing and using them.
After running the command pip install --upgrade google-cloud-logging, you will be able to authenticate with your GCP account, and use the Client Libraries.
Using them is as easy as importing the library with a command such as from google.cloud import logging, then instantiate a new client (which you can use by default, or even pass the Project ID and Credentials explicitly) and finally work with Logs as you want.
You may also want to visit the official library documentation, where you will find all the details of how to use the library, which methods and classes are available, and how to do most of the things, with lots of self-explanatory examples, and even comparisons between the different alternatives on how to interact with Stackdriver Logging.
As a small example, let me also share a snippet of how to retrieve the five most recent logs which have status more sever than "warning":
# Import the Google Cloud Python client library
from google.cloud import logging
from google.cloud.logging import DESCENDING
# Instantiate a client
logging_client = logging.Client(project = <PROJECT_ID>)
# Set the filter to apply to the logs, this one retrieves GAE logs from the default service with a severity higher than "warning"
FILTER = 'resource.type:gae_app and resource.labels.module_id:default and severity>=WARNING'
i = 0
# List the entries in DESCENDING order and applying the FILTER
for entry in logging_client.list_entries(order_by=DESCENDING, filter_=FILTER): # API call
print('{} - Severity: {}'.format(entry.timestamp, entry.severity))
if (i >= 5):
break
i += 1
Bear in mind that this is just a simple example, and that many things can be achieved using the Logging Client Library, so you should refer to the official documentation pages that I shared in order to get a more deep understanding of how everything works.
However it doesn't seem to have any impact on the output which is
always INFO for me.
add a logging handler, e.g.:
formatter = logging.Formatter('%(asctime)s %(process)d %(levelname)s: %(message)s')
consoleHandler = logging.StreamHandler()
consoleHandler.setLevel(logging.DEBUG)
consoleHandler.setFormatter(formatter)
logger.addHandler(consoleHandler)
Good Evening!
I've been looking into the possibility of using GAS(Google Apps Script) to host a small bit of javascript that lets me use the new Google finance apps api. The intention being that I'll be using the stock information for a project which involves the use of stock data. I know that there are a few ways to get stock information from Google, but the data that the finanace app returns is more in-line with other sources we are using. (One constraint on this project is that we have multiple sources).
I've written the javascript and I can call a httpc:request to the URL for the script given to me from Google. In the browser the JS returns the json object as I want it, however when the call is made from Erlang I'm getting it in a list of ascii. From checking the values it appears to be a document starting like:
Below is the javascript and the url to see the json:
https://script.google.com/macros/s/AKfycbzEvuuQl4jkrbPCz7hf9Zv4nvIOzqAkBxL1ixslLBxmSEhksQM/exec
function doGet() {
var stock = FinanceApp.getStockInfo('LON:TSCO');
return ContentService.createTextOutput(JSON.stringify(stock))
.setMimeType(ContentService.MimeType.JSON);
}
For the erlang, it's a simple request but I've not been doing erlang long, so perhaps I've messed something up here (The URL being the one mentioned above). I've got crypto / ssl / inets when I'm testing this on the command line.
{ok, {Version, Headers, Body}} = httpc:request(get, URL, []}, [], []).
I think it's also worth mentioning that when i curl it from Cygwin, I get a massive load of HTML also, I've included it below, but if you see it you'll thank me for not posting it in here! http://pastebin.com/UtJHXjRm
I've been updating the script as I go with the new versions but I'm at a bit of a loss as to why it's not returning correctly.
If anyone can give me any pointers I'd be very grateful! I get the feeling that it's not intended to be used this way, perhaps only within other Google products and such.
Cheers!
It would be necessary to review how are you deploying the Web App, specifically the Who has access to the app, to access without authentication should be configured as shown in the image:
See Deploying Your Script as a Web App from the documentation.
In my test, by running:
curl -L https://script.google.com/macros/s/************/exec
Get the following result:
{
"priceopen":358,
"change":2.199981689453125,
"high52":388.04998779296875,
"tradetime":"2013-10-11T15:35:18.000Z",
"currency":"GBX",
"timezone":"Europe/London",
"low52":307,
"quote":357.8999938964844,
"name":"Tesco PLC",
"exchange":"LON",
"marketcap":28929273763,
"symbol":"TSCO",
"volumedelay":0,
"shares":8083060703,
"pe":23.4719295501709,
"eps":0.15248000621795654,
"price":357.8999938964844,
"has_stock_data":true,
"volumeavg":14196534,
"volume":8885809,
"changepct":0.6184935569763184,
"high":359.5,
"datadelay":0,
"low":355.8999938964844,
"closeyest":355.70001220703125
}
Possibly your GET is not following the REDIRECT that happens when you use contentService. Look at the html returned there is a redirect in there.