Docker and git error while deploying to server - mysql

I was using a CI CD pipeline to deploy my project to the server.
However it suddenly stopped working and I got two errors.
The first one is related to git and
The second one is a docker error.
Can somebody help me what could be the problem?
32 out: Total reclaimed space: OB
33 err: error: cannot pull with rebase:
You have unstaged changes. err: error: please commit or stash them. 34 35
out: docker build -f Dockerfile . -t
tourmix-next
36 err: time="20***-10-08T11:06:33Z"
level-error msg="Can't add file
/mnt/tourmix-main/database/mysql.sock
to tar: archive/tar: sockets not supported"
37 out: Sending build context to Docker daemon 255MB
38
out: Step 1/21 : FROM node:1ts as
dependencies
39 out: Its: Pulling from library/node
40 out: Digest:
sha256:b35e76ba744a975b9a5428b6c3cde1a1 cf0be53b246e1e9a4874f87034***b5a
47 41 out: Status: Downloaded newer image for node:1ts
2 42 out: ---> 946ee375d0e0
3 4 out: Step 2/21: WORKDIR /tourmix out: ---> Using cache
5 45 out: ---> 05e933ce4fa7
This is my Dockerfile:
1 FROM node:1ts as dependencies
2 WORKDIR /tourmix
3 COPY package*.json ./
4 RUN npm install --force
5
6 FROM node:lts as builder
7 WORKDIR /tourmix
8 COPY . .
9 COPY -from-dependencies /tourmix/node_modules ./node_modules
10 RUN npx prisma generate
11 RUN npm run build
12
13 FROM node:lts as runner
14 WORKDIR /tourmix
15 ENV NODE_ENV production
16 # If you are using a custom next.config.js file, uncomment this line.
17 COPY --from-builder /tourmix/next.config.js ./
18 COPY --from-builder /tourmix/public ./public
19 COPY --from-builder /tourmix/.next ./.next
20 COPY --from-builder /tourmix/node_modules ./node_modules
21 COPY -from-builder /tourmix/package.json ./package.json
22 COPY --from-builder /tourmix/.env ./.env
24 # copy the prisma folder
25 EXPOSE 3000
26 CMD ["yarn", "start"]
This is my GitHub workflow file:
# This is a basic workflow that is manually triggered
name: Deploy application
# Controls when the action will run. Workflow runs when manually triggered using the UI
# or API.
on:
push:
branches: [master]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "greet"
deploy:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: multiple command
uses: appleboy/ssh-action#master
with:
host: ${{secrets.SSH_HOST}}
username: ${{ secrets. SSH_USER }}
key: ${{ secrets.SSH_PRIVATE_KEY }}
port: ${{ secrets.SSH_PORT}} passphrase: ${{ secrets.SSH_PASSPHRASE}}
script:|
docker system prune -a -f
cd /mnt/tourmix-main
git pull origin master --rebase
make release
docker system prune -a -f
- uses: actions/checkout#v3
with:
clean: 'true'

Start with the first error:
Add a git clean pre-step in your pipeline, to clean any private file from your workspace.
If you are using GitLab as a CICD platform, use Git clean flags (GitLab Runner 11.10+, Q2 2019)
For a GitHub Action, if the error is on the git pull command, add a git clean -ffdx just before the git pull.
script:|
docker system prune -a -f
cd /mnt/tourmix-main
git clean -ffdx <====
git stash <====
git pull origin master --rebase
make release
docker system prune -a -f

Related

running gui application on github hosted runner

for testing purposes, is it possible to run GUI applications on GitHub-hosted runners?
I tried to run Windows Calculator (Microsoft.WindowsCalculator_8wekyb3d8bbwe!App) on "windows-2022" via WinAppDriver and it fails with "WebDriverException: Package was not found".
Any suggestion(s)?
TIA,
Adrian.
P.S. here is my GitHub Actions workflow for the above:
# ISSUE fails with WebDriverException: Package was not found
# see https://github.com/QA-Automation-Starter/qa-automation/actions/runs/3234841483/jobs/5298454871
build-and-test-on-windows:
name: windows build&test
# see https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
runs-on: windows-2022
environment: development
steps:
- uses: actions/checkout#v2
- uses: actions/setup-java#v3
with:
java-version: '8'
distribution: 'temurin'
cache: maven
settings-path: ${{ github.workspace }}
# ISSUE somehow should run WinAppDriver
# see https://github.com/microsoft/WinAppDriver/issues/1722
# and https://github.com/actions/runner-images/blob/main/images/win/Windows2022-Readme.md
# TODO maybe, should re-publish the site from here (?)
- run: |
choco install -y autologon
autologon %USERNAME% $USERDOMAIN%
start cmd /c "C:\Program Files (x86)\Windows Application Driver\WinAppDriver.exe"
cd qa-testing-example
mvn install ^
-s %GITHUB_WORKSPACE%\settings.xml ^
-Pmode-build-fast,mode-build-quiet,environment-default,testing-windows,device-windows
shell: cmd

git-secrets decryption in GitHub actions

Hello Github actions community :)
I have a workflow in github actions that I don't quite understand why it is not working.
I am currently using git-secrets to encrypt my credentials using git-secrets and I am trying to decrypt them in the github actions workflow.
This is the code block that I execute when I want to decrypt the files:
- name: Reveal data
run: |
echo
echo 'Before decrypt'
ls -ls
git secret reveal -p ${{ secrets.PASSPHRASE }} -f
echo 'After decrypt'
ls -ls
git secret whoknows
Before decrypt
total 4
4 -rw-r--r-- 1 runner docker 630 Jul 18 09:39 secrets.md.secret
done. all 1 files are revealed.
After decrypt
total 4
4 -rw-r--r-- 1 runner docker 630 Jul 18 09:39 secrets.md.secret
testing#testing.com
According to github actions this works because as you can see the github actions returns 'done. all 1 files are revealed.'. However, as you can see below, no new file is being generated.
Locally it works and I get the decrypted file by running the same command.
How to reproduce it locally:
Install git-secrets
Create a GPG key (gpg --full-generate-key)
Run 'git secret tell email-used-in-the-gpg
Run 'git secret add filename
Run 'git secret hide' to encrypt the file
Run 'rm filename'
Run 'git secret reveal' and pass the password. This will create the decrypted file
How to reproduce it in github actions:
Create a new workflow
Paste this step:
- name: Reveal
run: |
git secret reveal -p ${{ secrets.PASSPHRASE }}
Does anyone have any idea what this is about? Github Workflows does not allow file creation maybe?
Thank you very much in advance and best regards!

Use case of OpenShift + buildConfig + ConfigMaps

I am trying to create and run a buildconfig yml file.
C:\OpenShift>oc version
Client Version: 4.5.31
Kubernetes Version: v1.18.3+65bd32d
Background:-
I have multiple Springboot WebUI applications which i need to deploy on OpenShift
To have separate set of config yml files ( image stream, buildconfig, deployconfig, service, routes),
for each and every application seems to be very inefficient.
Instead i would like to have a single set of parameterized yml files
to which i can pass on custom parameters to setup each individual application
Solution so far:-
Version One
Dockerfile-
FROM org/rhelImage
USER root
# Install Yum Packages
RUN yum -y install\
net-tools\
&& yum -y install nmap-ncat\
RUN curl -s --create-dirs --insecure -L ${ARTIFACTURL} -o ${APPPATH}/${ARTIFACT}
# Add docker-entrypoint.sh to the image
ADD docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod -Rf 775 /app && chmod 775 /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh
RUN chmod -R g+rx /app
# Expose port
EXPOSE $MY_PORT
# Set working directory when container starts
WORKDIR $APPPATH
# Starting the applicaiton using ENTRYPOINT
#ENTRYPOINT ["sh","/docker-entrypoint.sh"]
$ oc create configmap myapp-configmap --from-env-file=MyApp.properties
configmap/myapp-configmap created
$ oc describe cm myapp-configmap
Name: myapp-configmap
Namespace: 1234
Labels: <none>
Annotations: <none>
Data
====
APPPATH:
----
/app
ARTIFACT:
----
myapp.jar
ARTIFACTURL:
----
"https://myorg/1.2.3.4/myApp-1.2.3.4.jar"
MY_PORT:
----
12305
Events: <none>
buildconfig.yaml snippet
strategy:
dockerStrategy:
env:
- name: GIT_SSL_NO_VERIFY
value: "true"
- name: ARTIFACTURL
valueFrom:
configMapKeyRef:
name: "myapp-configmap"
key: ARTIFACTURL
- name: ARTIFACT
valueFrom:
configMapKeyRef:
name: "myapp-configmap"
key: ARTIFACT
This works fine. However I somehow need to have those env: variables in file.
I am doing this to have greater flexibility, i.e. lets say I have a new variable introduced in docker file, I need NOT change the buildconfig.yml
I just add the new key:value pair to the property file, rebuild and we are good to go
This is what I do next;
Version Two
Dockerfile
FROM org/rhelImage
USER root
# Install Yum Packages
RUN yum -y install\
net-tools\
&& yum -y install nmap-ncat\
#Intializing the variables file;
RUN ["sh", "-c", "source ./MyApp.properties"]
RUN curl -s --create-dirs --insecure -L ${ARTIFACTURL} -o ${APPPATH}/${ARTIFACT}
# Add docker-entrypoint.sh to the image
ADD docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod -Rf 775 /app && chmod 775 /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh
RUN chmod -R g+rx /app
# Expose port
EXPOSE $MY_PORT
# Set working directory when container starts
WORKDIR $APPPATH
# Starting the applicaiton using ENTRYPOINT
#ENTRYPOINT ["sh","/docker-entrypoint.sh"]
$ oc create configmap myapp-configmap --from-env-file=MyApp.properties=C:\MyRepo\MyTemplates\MyApp.properties
configmap/myapp-configmap created
C:\OpenShift>oc describe configmaps test-configmap
Name: myapp-configmap
Namespace: 1234
Labels: <none>
Annotations: <none>
Data
====
MyApp.properties:
----
APPPATH=/app
ARTIFACTURL="https://myorg/1.2.3.4/myApp-1.2.3.4.jar"
ARTIFACT=myapp.jar
MY_PORT=12035
Events: <none>
buildconfig.yaml snippet
source:
contextDir: "${param_source_contextdir}"
configMaps:
- configMap:
name: "${param_app_name}-configmap"
However the build fails
STEP 9: RUN ls ./MyApp.properties
ls: cannot access ./MyApp.properties: No such file or directory
error: build error: error building at STEP "RUN ls ./MyApp.properties": error while running runtime: exit status 2
This means that the config map file didnt get copy to folder.
Can you please suggest what to do next?
I think you are misunderstanding Openshift a bit.
The first thing you say is
To have separate set of config yml files ( image stream, buildconfig, deployconfig, service, routes), for each and every application seems to be very inefficient.
But that's how kubernetes/openshift works. If your resource files look the same, but only use a different git resource or image for example, then you probably are looking for Openshift Templates.
Instead i would like to have a single set of parameterized yml files to which i can pass on custom parameters to setup each individual application
Yep, I think Openshift Templates is what you are looking for. If you upload your template to the service catalog, whenever you have a new application to deploy, you can add some variables in a UI and click deploy.
An Openshift Template is just a parameterised file for all of your openshift resources (configmap, service, buildconfig, etc.).
If your application needs to be build from some git repo, using some credentials, you can parameterise those variables.
But also take a look at Openshift's Source-to-Image solution (I'm not sure what version you are using, so you'll have to google some resources). It can build and deploy your application without you having to write your own Resource files.

Command not found when SSHing into server via non-interactive session

I'm using a GitHub action to SSH into my staging server to pull the latest from the repo (which succeeds) then install node modules: yarn (which fails), build the app: yarn build:app (which fails), then restart the app: pm2 restart all (which fails). From what I've read from researching, the commands aren't found because the SSH into the server is a non-interactive session and many things aren't added to $PATH. I have tried adding export PATH="$PATH:/home/***/.nvm/versions/node/v14.5.0/bin/pm2:/home/***/.nvm/versions/node/v14.5.0/bin/pm2" to my script to no avail. I'm still getting command not found.
name: Test Deployment
on:
push:
branches:
- staging
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Deploy Staging
if: github.ref == 'refs/heads/staging'
uses: appleboy/ssh-action#master
with:
host: ${{ secrets.STAGING_SSH_HOST }}
username: ${{ secrets.STAGING_SSH_USERNAME }}
key: ${{ secrets.STAGING_SSH_KEY }}
passphrase: ${{ secrets.STAGING_SSH_PASSPHRASE }}
port: ${{ secrets.STAGING_SSH_PORT }}
script: |
export PATH="$PATH:/home/***/.nvm/versions/node/v14.5.0/bin/pm2:/home/***/.nvm/versions/node/v14.5.0/bin/pm2"
cd ~/***
git pull origin staging
yarn
yarn build:app
pm2 restart all
Receiving errors:
======CMD======
export PATH="$PATH:/home/***/.nvm/versions/node/v14.5.0/bin/pm2:/home/***/.nvm/versions/node/v14.5.0/bin/pm2"
cd ~/***
git pull origin staging
yarn
yarn build:app
pm2 restart all
======END======
err: From github.com:***/***
err: * branch staging -> FETCH_HEAD
err: *** staging -> origin/staging
out: Merge made by the 'recursive' strategy.
2021/04/16 21:28:17 Process exited with status 127
out: .github/workflows/main.yml | 2 +-
out: 1 file changed, 1 insertion(+), 1 deletion(-)
err: bash: line 3: yarn: command not found
err: bash: line 4: yarn: command not found
err: bash: line 5: pm2: command not found
Instead of
export path='$PATH:/home/***/.nvm/versions/node/v14.5.0/bin/pm2:/home/***/.nvm/versions/node/v14.5.0/bin/pm2'
you should try
export PATH="$PATH:/home/***/.nvm/versions/node/v14.5.0/bin/pm2:/home/***/.nvm/versions/node/v14.5.0/bin/pm2"
Notice:
PATH - uppercase
Use double quote instead of single quote to expand the previous value of PATH variable

Asp.net core + Aws Elastic Beanstalk + Bitbucket pipeline

How can we use bitbucket pipelines to update an asp.net core website on aws elastic beanstalk?
i know this is late answer but i did same thing few days ago so here is example how i did it
firstly you have to enable pipeline in bitbucket choose .NET CORE
in bitbucket-pipelines.yml you need yo write something like this:
image: microsoft/dotnet:sdk
pipelines:
branches:
staging:
- step:
name: build publish prepare and zip
caches:
- dotnetcore
script:
- apt-get update && apt-get install --yes zip
- export PROJECT_NAME=<your-project-name>
- dotnet restore
- dotnet build $PROJECT_NAME
- dotnet publish --self-contained --runtime win-x64 --configuration Release
- zip -j site.zip /opt/atlassian/pipelines/agent/build/<your-project-name>/bin/Release/netcoreapp2.0/win-x64/publish/* -x aws-windows-deployment-manifest.json
- zip -r -j application.zip site.zip /opt/atlassian/pipelines/agent/build/<your-project-name>/bin/Release/netcoreapp2.0/win-x64/publish/aws-windows-deployment-manifest.json
artifacts:
- application.zip
- step:
name: upload to elasticbeanstalk
script:
- pipe: atlassian/aws-elasticbeanstalk-deploy:0.5.0
variables:
APPLICATION_NAME: '<application-name>'
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
#COMMAND: 'upload-only'
ZIP_FILE: 'application.zip'
ENVIRONMENT_NAME: '<environment-name>'
WAIT: 'true'
in settings -> pipelines -> variables you have to set aws keys: access secret and region that will used by $ ($AWS_SECRET_ACCESS_KEY)
additionally you will have to create s3bucket "-elsticbeanstalk-deployments" (if you dont create it, when the environment will try to upload your zip it will show you error with name of bucket "not found" so just copy the name and create it in s3)