Spring in Kubernetes tries to reach DB at pod IP - mysql

Question

I'm facing an issue while deploying a Spring API which should connect to a MySQL database.
I am deploying a standalone MySQL using the [bitnami helm chart][1] with the following values:
primary:
service:
type: ClusterIP
persistence:
enabled: true
size: 3Gi
storageClass: ""
extraVolumes:
- name: mysql-passwords
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: mysql-spc
extraVolumeMounts:
- name: mysql-passwords
mountPath: "/vault/secrets"
readOnly: true
configuration: |-
[mysqld]
default_authentication_plugin=mysql_native_password
skip-name-resolve
explicit_defaults_for_timestamp
basedir=/opt/bitnami/mysql
plugin_dir=/opt/bitnami/mysql/lib/plugin
port=3306
socket=/opt/bitnami/mysql/tmp/mysql.sock
datadir=/bitnami/mysql/data
tmpdir=/opt/bitnami/mysql/tmp
max_allowed_packet=16M
bind-address=0.0.0.0
pid-file=/opt/bitnami/mysql/tmp/mysqld.pid
log-error=/opt/bitnami/mysql/logs/mysqld.log
character-set-server=UTF8
collation-server=utf8_general_ci
slow_query_log=0
slow_query_log_file=/opt/bitnami/mysql/logs/mysqld.log
long_query_time=10.0
[client]
port=3306
socket=/opt/bitnami/mysql/tmp/mysql.sock
default-character-set=UTF8
plugin_dir=/opt/bitnami/mysql/lib/plugin
[manager]
port=3306
socket=/opt/bitnami/mysql/tmp/mysql.sock
pid-file=/opt/bitnami/mysql/tmp/mysqld.pid
auth:
createDatabase: true
database: api-db
username: api
usePasswordFiles: true
customPasswordFiles:
root: /vault/secrets/db-root-pwd
user: /vault/secrets/db-pwd
replicator: /vault/secrets/db-replica-pwd
serviceAccount:
create: false
name: social-app
I use the following deployment which runs a spring API (with Vault secret injection):
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: social-api
name: social-api
spec:
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
selector:
matchLabels:
app: social-api
template:
metadata:
labels:
app: social-api
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: 'social'
spec:
serviceAccountName: social-app
containers:
- image: quay.io/paulbarrie7/social-network-api
name: social-network-api
command:
- java
args:
- -jar
- "-DSPRING_DATASOURCE_URL=jdbc:mysql://social-mysql.default.svc.cluster.local/api-db?useSSL=false"
- "-DSPRING_DATASOURCE_USERNAME=api"
- "-DSPRING_DATASOURCE_PASSWORD=$(cat /secrets/db-pwd)"
- "-DJWT_SECRET=$(cat /secrets/jwt-secret)"
- "-DS3_BUCKET=$(cat /secrets/s3-bucket)"
- -Dlogging.level.root=DEBUG
- -Dspring.datasource.hikari.maximum-pool-size=5
- -Dlogging.level.com.zaxxer.hikari.HikariConfig=DEBUG
- -Dlogging.level.com.zaxxer.hikari=TRACE
- social-network-api-1.0-SNAPSHOT.jar
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 100m
memory: 100Mi
ports:
- containerPort: 8080
volumeMounts:
- name: aws-credentials
mountPath: "/root/.aws"
readOnly: true
- name: java-secrets
mountPath: "/secrets"
readOnly: true
volumes:
- name: aws-credentials
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: aws-secret-spc
- name: java-secrets
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: java-spc
Identifier are ok, when I run an interactive mysql pod I can connect to the database. However name resolution for the Spring API is wrong since I get the error:
java.sql.SQLException: Access denied for user 'api'#'10.24.0.194' (using password: YES)
which is wrong since 10.24.0.194 is the API pod address and not the mysql pod or service address, and I cant solve why.
Any idea?
[1]: https://artifacthub.io/packages/helm/bitnami/mysql

Answer 1

Thanks to David's suggestion I succeeded in solving my problem.
Actually there were two issues in my configs.
First the secrets were indeed misinterpreted, then I've changed my command/args to:
command:
- "/bin/sh"
- "-c"
args:
- |
DB_USER=$(cat /secrets/db-user)
DB_PWD=$(cat /secrets/db-pwd)
JWT=$(cat /secrets/jwt-secret)
BUCKET=$(cat /secrets/s3-bucket)
java -jar \
-DSPRING_DATASOURCE_URL=jdbc:mysql://social-mysql.default.svc.cluster.local/api-db?useSSL=false \
"-DSPRING_DATASOURCE_USERNAME=$DB_USER" \
"-DSPRING_DATASOURCE_PASSWORD=$DB_PWD" \
"-DJWT_SECRET=$JWT" \
"-DS3_BUCKET=$BUCKET" \
-Dlogging.level.root=DEBUG \
social-network-api-1.0-SNAPSHOT.jar
And the memory resources set were also too low, so I have changed them to:
resources:
limits:
cpu: 100m
memory: 400Mi
requests:
cpu: 100m
memory: 400Mi