Streamlit application reloads every 30secs when deployed on kubernetes - kubernetes-ingress

Question

Hi I have deployed a streamlit application which acts as a UI for downloading data from our platform. After deploying on kubernetes I observe that application keeps reloading after every 30 seconds, which is very annoying.
If I access the app by port forwarding the service it works ok but somehow using via nginx, it has the above mentioned problem.
Has anyone faced this issue ?
I was looking into the streamlit forum and saw same issue like mine but there is no clear solution.
Streamlit reruns all 30 seconds
I do see a websocket time of ~30s in developer tool under network section in browser.
My k8s manifest file is as follows
apiVersion: apps/v1
kind: Deployment
metadata:
name: streamlit-deployment
labels:
app: streamlit
spec:
replicas: 1
selector:
matchLabels:
app: streamlit
template:
metadata:
labels:
app: streamlit
spec:
containers:
- name: streamlit
image: <image>:<tag>
imagePullPolicy: Always
ports:
- containerPort: 8501
livenessProbe:
httpGet:
path: /healthz
port: 8501
scheme: HTTP
timeoutSeconds: 1
readinessProbe:
httpGet:
path: /healthz
port: 8501
scheme: HTTP
timeoutSeconds: 1
resources:
limits:
cpu: 1
memory: 2Gi
requests:
cpu: 100m
memory: 745Mi
volumeMounts:
- name: data
mountPath: /data
volumes:
- name: data
persistentVolumeClaim:
claimName: data
---
apiVersion: v1
kind: Service
metadata:
labels:
app: streamlit
name: streamlit-service
spec:
ports:
- nodePort: 32640
port: 8501
protocol: TCP
targetPort: 8501
selector:
app: streamlit
sessionAffinity: None
type: NodePort
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: webapp
labels:
app: streamlit
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /webapp(/|$)(.*)
pathType: Prefix
backend:
service:
name: streamlit-service
port:
number: 8501
`
I tried to add following annotation in nginx definition but it didnt work
nginx.ingress.kubernetes.io/connection-draining: "true" nginx.ingress.kubernetes.io/connection-draining-timeout: "3000"
Also I looked into the source code and was wondering if its because of tornado settings in
lib/streamlit/web/server/server.py which has web socket ping timeout set as 30 seconds.
"websocket_ping_timeout": 30,
I tried to set this to a higher value and create a new image for my deployment, but unfortunately it didnt help.
I'd really appreciate any leads.

Answer 1

Finally, we found out that issue was due to the default time out of the GCP load balancer, which was causing the web socket connection to re-initialize every 30 secs. All we needed to do was change the time out in the backend configuration of the load balancer. I hope it helps.