I have deployed my 2 java applications( ant and maven based) in gcp compute engine , i wish to connect those log files to GCP stackdriver logging .
Can any one help me on this out ?
You need to configure the logging agent, see here: https://cloud.google.com/logging/docs/agent/installation
You can use the stackdriver logging library for java: https://cloud.google.com/logging/docs/setup/java
That being said, if all you need to run is some java applications, it probably makes more sense to deploy them in appengine standard (or flex, depending on needs), where logging is configured out of the box.
Related
We have a use case to monitor kubernetes clusters and I am trying to find the list of exceptions thrown by kubernetes to reflect the status of the k8s server (in a namespace) while trying to submit a job on the UI.
Example: if k8s server throws ClusterNotFound exception that means we cannot submit any more jobs to that api server.
Is there such a comprehensive list?
I came across this in Go Lang. Will this be it? Does java has something like this?
The file you are referencing is a part of Kubernetes library used by many Kubernetes components for API requests fields validations. As all Kubernetes components are written in Go and I couldn't find any plans to port Kubernetes to Java, it's unlikely to have a Java version of that file.
However, there is an officially supported Kubernetes client library, written in Java, so you can check for the proper modules to validate API requests and process API responses in the java-client repostiory or on the javadoc site.
For example, objects that are used to contain proper or improper HTTP replies from Kubernetes apiserver: V1Status and ApiExceptions, (repository link)
Please consider to check java-client usage examples for better understanding.
Detailed Kubernetes RESTful API reference could be found on the official page
For example: Deployment create request
If you are really interested in Kubernetes cluster monitoring and logging aspects, please consider to read the following articles at the beginning:
Metrics For Kubernetes System Components
Kubernetes Control Plane monitoring with Datadog
How to monitor Kubernetes control plane
Logging Architecture
A Practical Guide to Kubernetes Logging
I have a Google Firebase app with a Cloud Functions back-end. I'm using the node.js 10 runtime which is Ubuntu 18.04. Users can upload files to Google Cloud Storage, and that triggers a GCF function.
What I'd like that function to do is copy that file to Google Cloud Filestore (that's File with an L), Google's new NFS-mountable file server. They say the usual way to do that is from the command line with gsutil rsync or gcloud compute scp.
My question is: are either or both of those commands available on GCF nodes? Or is there another way? (It would be awesome if I could just mount the Filestore share on the GCF instance, but I'm guessing that's nontrivial.)
Using NFS based storage is not a good idea in this environment. NFS works by providing a mountable file system, something that will not work in Cloud Functions environment as the file system is read only with the exception of /tmp folder.
You should consider using cloud native storage systems like GCS for which the Application Default Credentials are already setup. See the list of supported services here.
According to the official documentation Cloud Filestore documentation
Use Cloud Filestore to create fully managed NFS file servers on Google
Cloud Platform (GCP) for use with applications running on Compute
Engine virtual machines (VMs) instances or Google Kubernetes Engine
clusters.
You can not mount the Filestore on GCF.
Also, you can not execute gsutil or gcloud commands from a Google cloud function Writing Cloud Functions.
Google Cloud Functions can be written in Node.js, Python, and Go, and
are executed in language-specific runtimes
I came across the article : Bringing the best of serverless to you
where I came to know about upcoming product called Serverless containers on Cloud Functions which is currently in Alpha.
As described in the article:
Today, we’re also introducing serverless containers, which allow you
to run container-based workloads in a fully managed environment and
still only pay for what you use.
and in GCP solutions page
Serverless containers on Cloud Functions enables you to run your own containerized workloads on
GCP with all the benefits of serverless. And you will still pay only
for what you use. If you are interested in learning more about
serverless containers, please sign up for the alpha.
So my question is how this serverless containers different from app engine flexible with custom runtime, which also use a docker file?
And it's my suspicion, since mentioned named is Serverless containers on Cloud Functions, the differentiation may include role of cloud functions. If so what is the role played by cloud functions in the serverless containers?
Please clarify.
What are Cloud Funtions?
From the official documentation:
Google Cloud Functions is a serverless execution environment for building and connecting cloud services. With Cloud Functions you write simple, single-purpose functions that are attached to events emitted from your cloud infrastructure and services. Your function is triggered when an event being watched is fired. Your code executes in a fully managed environment. There is no need to provision any infrastructure or worry about managing any servers.
In simple words, the Cloud Function is triggered by some event (HTTP request, PubSub message, Cloud Storage file insert...), runs the code of the function, returns a result and then the function dies.
Currently there are available four runtime environments:
Node.js 6
Node.js 8 (Beta)
Python (Beta)
Go (Beta)
With the Serverless containers on Cloud Functions product it is intended that you can provide your own custom runtime environment with a Docker Image. But the life cycle of the Cloud Function will be the same:
It is triggered > Runs > Outputs Result > Dies
App Engine Flex applications
Applications running in the App Engine flexible environment are deployed to virtual machines, i.e Google Cloud Compute Engine instances. You can choose the type of machine you want use and the resources (CPU, RAM, disk space). The App Engine flexible environment automatically scales your app up and down while balancing the load.
As well as in the case of the Cloud Functions there runtimes provided by Google but if you would like to use an alternative implementation of Python, Java, Node.js, Go, Ruby, PHP, .NET you can use Custom Runtimes. Or even you can work with another language like C++, Dart..., you just need to provide a Docker Image for your Application.
What are differences between Cloud Functions and App Engine Flex apps?
The main difference between them are its life cycle and the use case.
As commented above a Cloud Function has a defined life cycle and it dies when it task concludes. They should be used to do 1 thing and do it well.
On the other hand an Application running on the GAE Flex environment will always have at least 1 instance running. The typical case for this applications are to serve several endpoints where users can do REST API calls. But they provide more flexibility as you have full control over the Docker Image provided. You can do "almost" whatever you want there.
What is a Serverless Container?
As stated on the official blog post (search for Serverless Containerss), it's basically a Cloud Function running inside a custom environment defined by the Dockerfile.
It is stated on the official blog post:
With serverless containers, we are providing the same underlying
infrastructure that powers Cloud Functions, but you’ll be able to
simply provide a Docker image as input.
So, instead of deploying your code on the CF, you could also just deploy the Docker image with the runtime and the code to execute.
What's the difference between this Cloud Functions with custom runtimes vs App Engine Flexible?
There are 5 basic differences:
Network: On GAE Flexible you can customize the network the instances run. This let's you add firewalls rules to restrict egress and ingress traffic, block specific ports or specify the SSL you wish to run.
Time-Out: Cloud Functions can run for a maximum of 9 minutes, Flexible on the other hand, can run indefinitely.
Ready only environment: Cloud Functions environment is read-only while Flexible could be written (this is only intended to store spontaneous information as once the Flexible instance is restarted or terminated, all the stored data is lost).
Cold Boot: Cloud Functions are fast to deploy and fast to start compared to Flexible. This is because Flexible runs inside a VM, thus this extra time is taken in order for the VM to start.
How they work: Cloud functions are event driven (ex: upload of photo to cloud storage executing a function) on the other hand flexible is request driven.(ex: handling a request coming from a browser)
As you can see, been able to deploy a small amount of code without having to take care of all the things listed above is a feature.
Also, take into account that Serverless Containers are still in Alpha, thus, many things could change in the future and there is still not a lot of documentation explaining in-depth it's behavior.
I'm working on an architecture to deploy my webapp. I would like to use Google Managed Instance Groups because I have some strict requirements. I was wondering:
which is the best Web container to be deployed in a distributed environment?
I'm familiar with Tomcat, it's Tomcat OK to be deployed in an instance group?
my Webapp running on tomcat will generate logs that will be stored in the current machine hosting tomcat. How should I handle distributed application logs.
I don't want to lose information and I would like to have a single view of all log of my webapp even if distributed, Is it that possible?
Thanks
I have used tomcat in GCP for over a year and it has worked without problems with the load balancer. To solve the issue of the logs you must use an agent to save the logs in stackdriver https://cloud.google.com/logging/docs/view/service/agent-logs
I have GCE instances and provide our own Web APIs on the VMs. I really want to use GCP Cloud Trace to check the latency. Can I use it on GCE? Some say that it works on GAE. As far as I've read docs, it does not seem that I can use it.