logs disappear in cloud run once I start using google's LoggingAppender in logback? - logback

Question

I have no idea how to debug since there is no errors. I have this logback file but this made all the logs disappear for my cloudrun instance EXCEPT the system.out stuff that logback logs on startup..
<?xml version="1.0" encoding="UTF-8" ?>
<configuration scan="true" scanPeriod="30 seconds" debug="true">
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
<appender name="CLOUD" class="com.google.cloud.logging.logback.LoggingAppender">
<!-- Optional : filter logs at or above a level -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<flushLevel>WARN</flushLevel> <!-- Optional : default ERROR -->
</appender>
<appender name="ASYNC-SERVERLOG" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="CLOUD" />
</appender>
<!-- TBD for later use for apps that use log4jdbc it doesn't hurt apps that
don't use it -->
<logger name="jdbc.sqlonly" level="INFO" />
<logger name="jdbc.sqltiming" level="WARN" />
<logger name="jdbc.audit" level="WARN" />
<logger name="jdbc.resultset" level="WARN" />
<logger name="jdbc.connection" level="WARN" />
<root>
<level value="INFO" />
<appender-ref ref="ASYNC-SERVERLOG" />
</root>
</configuration>
Any ideas why logging stops working or where the logging is going to? Now that I changed logging to this, logging seems to go to the void :(.
I also tried out this post which logs to ConsoleAppender but NOTHING shows up in the json format although it is doing a line for EACH log statement I do which is really nice
GKE & Stackdriver: Java logback logging format?
I then tried this approach which is using FileAppender
How do I map my java app logging events to corresponding cloud logging event levels in GCP Felexible non-compat App Engine?
My logback for this one was
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>/var/log/app.log</file>
<append>true</append>
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="com.orderlyhealth.GCPCloudLoggingJSONLayout">
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg</pattern>
</layout>
</encoder>
</appender>
<appender name="ASYNC-SERVERLOG" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="FILE" />
</appender>
Any ideas?

Answer 1

The Java logging instructions in the documentation should help: https://cloud.google.com/run/docs/logging#run_manual_logging-java
In Cloud Run, output to Stdout and Stderr are forwarded on to Stackdriver. Logs which are recorded in files are lost.
I'm not sure the implications of the ASYNC-SERVERLOG appender, but on Cloud Run there is only CPU available to process and send logs while a request is being processed, so if something is asynchronously sending logs it will only work during that request or a future request that reaches the same container instance.