docker pentaho mysql driver issue - mysql

Question

I'm using Docker on Windows 10 to create a pentaho and mysql image that will run as containers on a network I define with docker network create.
The intention is that (as a first step) I will run a .KTR file with pan.sh that will read DB connection parameters from a .csv file and place these into the environment;
Get the DB connection parameters
Next a second .KTR checks to see if the DB exists using the above environment params;
Check DB exists
The problem is when I "Spin up" my project with docker-compose, step two fails with a driver not found issue. I've placed the drivers I require in the pentaho container's lib dir but I'm guessing this is not correct?
Ultimately, the intention is for a transformation to occur where data read from an OpenEdge DB is process via a series of steps in pentaho and written to the mysql DB.
Here's the supporting files;
Dockerfile;
FROM java:8-jre
MAINTAINER M Beynon
# Set required environment vars
ENV PDI_RELEASE=7.1 \
PDI_VERSION=7.1.0.0-12 \
CARTE_PORT=8181 \
PENTAHO_JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 \
PENTAHO_HOME=/home/pentaho
# Create user
RUN mkdir ${PENTAHO_HOME} && \
groupadd -r pentaho && \
useradd -s /bin/bash -d ${PENTAHO_HOME} -r -g pentaho pentaho && \
chown pentaho:pentaho ${PENTAHO_HOME}
# Add files
RUN mkdir $PENTAHO_HOME/docker-entrypoint.d
COPY docker-entrypoint.sh $PENTAHO_HOME/scripts/
RUN chown -R pentaho:pentaho $PENTAHO_HOME
RUN apt-get update && apt-get install -y libwebkitgtk-1.0-0
RUN apt-get update && apt-get install -y dos2unix
RUN dos2unix $PENTAHO_HOME/scripts/docker-entrypoint.sh && apt-get --purge remove -y dos2unix && rm -rf /var/lib/apt/lists/*
# Switch to the pentaho user
USER pentaho
# Download PDI
RUN /usr/bin/wget \
--progress=dot:giga \
http://downloads.sourceforge.net/project/pentaho/Data%20Integration/${PDI_RELEASE}/pdi-ce-${PDI_VERSION}.zip \
-O /tmp/pdi-ce-${PDI_VERSION}.zip && \
/usr/bin/unzip -q /tmp/pdi-ce-${PDI_VERSION}.zip -d $PENTAHO_HOME && \
rm /tmp/pdi-ce-${PDI_VERSION}.zip
ENV KETTLE_HOME=$PENTAHO_HOME/data-integration \
PATH=$KETTLE_HOME:$PATH
WORKDIR $KETTLE_HOME
ENTRYPOINT ["../scripts/docker-entrypoint.sh"]
The entrypoint;
#!/bin/bash
# based on https://github.com/aloysius-lim/docker-pentaho-di/blob/master/docker/Dockerfile
#exit script if any command fails (non-zero value)
set -e
cd resources
cp mysql-connector-java-5.1.42-bin.jar ../lib/
cp PROGRESS_DATADIRECT_JDBC_OE_ALL.jar ../lib
cd ../
echo 'Drivers copied!'
echo ''
echo 'Running transformation!'
#run a transformation (get db credentials)
./pan.sh -file=resources/Read-DBs.ktr
#run a transformation (does the db exist)
./pan.sh -file=resources/GoldBi-Exists.ktr
#redirect input variables
exec "$#"
The Docker compose file;
version: "2"
services:
db:
image: mysql:latest
networks:
- my-pdi-network
environment:
- MYSQL_ROOT_PASSWORD=tbitter
- MYSQL_DATABASE=mysql-db
ports:
- "3307:3306"
volumes:
- ./goldbi:/var/lib/mysql
pdi:
image: my-pdi-image:latest
networks:
- my-pdi-network
volumes:
- C:\Docker-Pentaho\resource:/home/pentaho/data-integration/resources
networks:
my-pdi-network:
The error coming from pentaho;
2017/05/30 15:28:56 - Table exists.0 - Error occurred while trying to connect to the database
2017/05/30 15:28:56 - Table exists.0 -
2017/05/30 15:28:56 - Table exists.0 - Error connecting to database: (using class org.gjt.mm.mysql.Driver)
2017/05/30 15:28:56 - Table exists.0 - Communications link failure
2017/05/30 15:28:56 - Table exists.0 -
2017/05/30 15:28:56 - Table exists.0 - The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
Many Thanks.
P.S. Does anyone know how to prevent build from rebuilding everything, even if it's only a small change to the dockerfile or entrypoint file?

Answer 1

I seem to have found the solution;
There were two issues. The first seems to be that the ENV vars set in the first transformation are not being utilised in the second transformation. The second was that the host name was wrong in the second transformation (DB-Exists). It should have been 'db' which is the name specified for the container in the docker-compose file. As the contains are both running on a custom network I'd specified, they can automatically 'talk' to each other via their service names...reverse DNS?