Git lab preparing CI scripts to deploy automatically to server in yii2? - yii2

Question

Here is my readme file to installation.I want to merge it on server using Ci in gitlab.
Installation
Clone project from Gitlab
git clone --single-branch --branch branchname https://gitlab.com/project/project.git
Update needed libraries
cd project
composer update
Go to project root directory cloned above and initialize all the configuration files which will be added to the application with a specific environment. Enter Yes to all the overwrite files ([Yes|No|All|Quit])
php init --env=Development
Create a new database, copy the file named common/config/main-local.example to common/config/main-local.php and adjust the components['db'] configuration in common/config/main-local.php accordingly.
cp common/config/main-local.example common/config/main-local.php
nano common/config/main-local.php
Change mail configuration inside common/config/main-local.php
Get Recaptcha keys (v2) from https://www.google.com/recaptcha/admin
Copy the param file named common/config/params.example to common/config/params.php and adjust the configuration in common/config/params.php accordingly.
cp common/config/params.example common/config/params.php
nano common/config/params.php
Apply migrations which will create tables needed for the application to work. Enter Yes to all prompts to dev environment but take extra care on the production environment (so it run the migrations and add the database tables to your db)
php yii migrate
Create assets folder under backend/web , allow group users to change the content and make the directory owned by apache user
mkdir -p backend/web/assets
chmod 775 backend/web/assets
sudo chown www-data:www-data backend/web/assets
What should be in the .gitlab-ci.yml file and ci/docker_install.sh
,ci/shell-scripts-dev.sh,ci/shell-scripts-prod.sh. Please also explain
how it work?

Answer 1

Here is you .gitlab-ci.yml file
#
# File is "indented" using multiple of 4 spaces
#
# Specify the docker image to use (only used if using docker runners)
# See: http://doc.gitlab.com/ee/ci/docker/using_docker_images.html
# From https://hub.docker.com/r/kaffineaddict/gitlabcakephp3/ - we could use image: kaffineaddict/gitlabcakephp3
image: php:7.2
# The docker services to configure
# See: http://doc.gitlab.com/ee/ci/docker/using_docker_images.html#what-is-service
services:
#- mysql:5.7
# Define custom build variables
# For the default gitlab variables see: http://doc.gitlab.com/ce/ci/variables/README.html
# These can be used below, or they will also be ENV variables available within' any scripts
# you execute from the CI
variables:
#MYSQL_DATABASE: site_zoova
#MYSQL_ALLOW_EMPTY_PASSWORD: "1"
#MYSQL_ROOT_PASSWORD: ThisIsAStrongPassword#^2
# Define commands that run before each job's script
before_script:
- umask 022 # set permissions to default directory permissions of 755 and default file permissions are 644,
# Install ssh-agent if not already installed, it is required by Docker.
# (change apt-get to yum if you use a CentOS-based image)
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
#- docker-php-ext-install pdo pdo_mysql mysqli
# Run ssh-agent (inside the build environment)
- eval $(ssh-agent -s)
# For Docker builds disable host key checking. Be aware that by adding that
# you are suspectible to man-in-the-middle attacks.
# WARNING: Use this only with the Docker executor, if you use it with shell
# you will overwrite your user's SSH config.
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
# Prepare the build environment. A way to overcome this is to create a script which installs all prerequisites prior the actual testing is done.
- bash ci/docker_install.sh > /dev/null
#- bash ci/docker_install.sh
# Prepare the build environment. A way to overcome this is to create a script which installs all prerequisites prior the actual testing is done.
- cd $CI_PROJECT_DIR && curl --silent --show-error https://getcomposer.org/installer | php
# Install composer dependencies
#- composer install --no-plugins --no-scripts
#- cd $CI_PROJECT_DIR && php composer.phar config cache-files-dir /cache/composer
- cd $CI_PROJECT_DIR && php composer.phar install --no-plugins --no-scripts --optimize-autoloader
- cd $CI_PROJECT_DIR
# Folder and file manipulation
- 'which rsync || ( apt-get install rsync -y )'
- '[ -d $CI_PROJECT_DIR/backend/web/assets ] || mkdir -p $CI_PROJECT_DIR/backend/web/assets'
## please change this line to the cache directories of yii mvc
# - rm -rf $CI_PROJECT_DIR/tmp/*
- rm -rf $CI_PROJECT_DIR/backend/web/assets/*
- rm -rf $CI_PROJECT_DIR/backend/runtime/debug/*
- rm -rf $CI_PROJECT_DIR/backend/runtime/logs/*
- rm -rf $CI_PROJECT_DIR/backend/runtime/mail/*
- rm -rf $CI_PROJECT_DIR/backend/runtime/URI/*
# Rest runtime files
- rm -rf $CI_PROJECT_DIR/rest/runtime/cache/*
- rm -rf $CI_PROJECT_DIR/backend/runtime/debug/*
- rm -rf $CI_PROJECT_DIR/backend/runtime/logs/*
- rm -rf $CI_PROJECT_DIR/backend/runtime/mail/*
- find $CI_PROJECT_DIR -type d -exec chmod 0755 {} \;
- find $CI_PROJECT_DIR -type f -exec chmod 0644 {} \;
- chmod -R 777 $CI_PROJECT_DIR/backend/web/assets
- chmod +x $CI_PROJECT_DIR/ci/*
- chown www-data:www-data $CI_PROJECT_DIR/ -R
# Make sure these dirs/files are not writable
# setup application
- chmod go-w $CI_PROJECT_DIR
#run code sniffer
- php ./vendor/bin/phpcs --encoding=utf-8 --extensions=php backend --colors
- php ./vendor/bin/phpcs --encoding=utf-8 --extensions=php common --colors
- php ./vendor/bin/phpcs --encoding=utf-8 --extensions=php rest --colors
#- |
# following will done once on the server
# cp common/config/main-local.example common/config/main-local.php
# cp common/config/params.example common/config/params.php
# cp rest/web/index.example rest/web/index.php
# genetraing files and added to z_rsync_exclude_list
#php ./init --env=Development --overwrite=All
# replacing credentials in config file
#sed -i "s/{APP_DB_HOST}/${APP_DB_HOST}/g" ${CI_PROJECT_DIR}/common/config/main-local.php
#sed -i "s/{APP_DB_NAME}/${APP_DB_NAME}/g" ${CI_PROJECT_DIR}/common/config/main-local.php
#sed -i "s/{APP_DB_USERNAME}/${APP_DB_USERNAME}/g" ${CI_PROJECT_DIR}/common/config/main-local.php
#sed -i "s/{APP_DB_PASSWORD}/${APP_DB_PASSWORD}/g" ${CI_PROJECT_DIR}/common/config/main-local.php
# updating composer for dependencies
#php composer.phar update
# running yii migrations
#yes | php yii migrate
# resolve errors through phpcs
#php ./vendor/bin/phpcs --encoding=utf-8 --extensions=php backend --colors
#php ./vendor/bin/phpcs --encoding=utf-8 --extensions=php common --colors
#php ./vendor/bin/phpcs --encoding=utf-8 --extensions=php rest --colors
# making required directories & assigning permissions
#mkdir -p backend/web/assets
#chmod 775 backend/web/assets
#chown www-data:www-data backend/web/assets
#- |
# Define commands that run before after all builds
after_script:
#- find . -type d -exec chmod 0755 {} \; # Set directory permissions #moved in each stage
#- find . -type f -exec chmod 0644 {} \; # Set file permissions #moved in each stage
# Define list of files that should be cached between subsequent runs -
# Composer stores all downloaded packages in the vendor/ directory.
# temporary commented out - builds failed
cache:
paths:
- vendor/
# stages is used to define build stages that can be used by jobs
# The specification of stages allows for having flexible multi stage pipelines
# The next stage only executes if all elements of the previous stage succeed
# Typically used for compiled languages testing and/or to automate deployments
stages:
- development
- production
#
# Run test on all branches but master
#
development:
stage: development
only:
- dev
script:
- echo Running dev
# DO NOT COPY THIS KEY TO PUBLIC PLACES
- ssh-add <(echo "$SSH_PRIVATE_KEY_DEV")
#- echo Running tests...
# Ex: - phpunit --configuration phpunit_myapp.xml
#- vendor/bin/phpunit # TODO: uncomment me - in docker file we have it as /usr/local/bin/phpunit
# Sync the files to the server
- echo Using rsync to push changes to dev server...
- rsync -ap --stats -e "ssh -p $SSH_PORT_DEV" --exclude-from 'ci/z_rsync_exclude_list.txt' $CI_PROJECT_DIR/ $SSH_USER_DEV#$SSH_IP_DEV:$PROJECT_PATH_DEV
- echo Running shell scripts on remote server
- ssh -t -p $SSH_PORT_DEV $SSH_USER_DEV#$SSH_IP_DEV 'cd '"'$PROJECT_PATH_DEV'"';ci/shell-scripts-dev.sh'
# Done
- echo Done pushing changes to dev...
#Environment is used to define that a job deploys to a specific environment. This allows easy tracking of all deployments to your environments straight from GitLab.
#If environment is specified and no environment under that name exists, a new one will be created automatically.
environment: development
#Make sure we don't push to live if build has failed
allow_failure: false #default behaviour
#
# Send to live server if branch is master
#
production:
stage: production
only:
- master
script:
- echo Running prod
# DO NOT COPY THIS KEY TO PUBLIC PLACES
- ssh-add <(echo "$SSH_PRIVATE_KEY_PROD")
#- echo Running tests...
# Ex: - phpunit --configuration phpunit_myapp.xml
#- vendor/bin/phpunit #TODO: uncomment me - in docker file we have it as /usr/local/bin/phpunit
# Push to live server now
- echo Using rsync to push changes to live server...
- rsync -ap --stats -e "ssh -p $SSH_PORT_PROD" --exclude-from 'ci/z_rsync_exclude_list.txt' $CI_PROJECT_DIR/ $SSH_USER_PROD#$SSH_IP_PROD:$PROJECT_PATH_PROD
- echo Running shell scripts on remote server
- ssh -t -p $SSH_PORT_PROD $SSH_USER_PROD#$SSH_IP_PROD 'cd '"'$PROJECT_PATH_PROD'"';ci/shell-scripts-prod.sh'
# Done
- echo Done pushing changes to live...
#Environment is used to define that a job deploys to a specific environment. This allows easy tracking of all deployments to your environments straight from GitLab.
#If environment is specified and no environment under that name exists, a new one will be created automatically.
environment: production
##make sure development built before moving to production
dependencies:
- development
Here you ci/docker_install.sh file
#!/bin/bash
# I had to specify DEBIAN_FRONTEND=noninteractive as we get after install apt-utils: debconf: unable to initialize frontend: Dialog
# We need to install dependencies only for Docker
[[ ! -e /.dockerenv ]] && [[ ! -e /.dockerinit ]] && exit 0
set -xe
# add the add-apt-repository command
DEBIAN_FRONTEND=noninteractive apt-get install software-properties-common -y
# Install git (the php image doesn't have it) which is required by composer
apt-get update -yqq
#debconf: delaying package configuration, since apt-utils is not installed
#apt-get install git -yqq
DEBIAN_FRONTEND=noninteractive apt-get install apt-utils git zip unzip -y
# Here you can install any other extension that you need
#docker-php-ext-install pdo_mysql intl mcrypt soap
docker-php-ext-install bcmath
Here your ci/shell-scripts-dev.sh file
#!/bin/bash
#do not enter current dir
#cd $(dirname $0)
BASEDIR=$(dirname "$0")
printf "\n"
printf "###############################################################################\n"
printf "# Running dev script from directory #\n"
printf "###############################################################################\n"
pwd
printf "\n"
printf "###############################################################################\n"
printf "# Create initial/ needed files #\n"
printf "###############################################################################\n"
# backend init files
if [ ! -f backend/config/main-local.php ]; then
printf "File backend/config/main-local.php does not exist, creating it, please edit configuration on server!\n"
cp backend/config/main-local.example backend/config/main-local.php
fi
if [ ! -f backend/config/params.php ]; then
printf "File backend/config/params.php does not exist, creating it, please edit configuration on server!\n"
cp backend/config/params.example backend/config/params.php
fi
if [ ! -f backend/web/index.php ]; then
printf "File backend/web/index.php does not exist, creating it, please edit configuration on server!\n"
cp backend/web/index.example backend/web/index.php
fi
if [ ! -f backend/web/index-test.php ]; then
printf "File backend/web/index-test.php does not exist, creating it, please edit configuration on server!\n"
cp backend/web/index.example backend/web/index-test.php
fi
# common init files
if [ ! -f common/config/main.php ]; then
printf "File common/config/main.php does not exist, creating it, please edit configuration on server!\n"
cp common/config/main.example common/config/main.php
fi
if [ ! -f common/config/main-local.php ]; then
printf "File common/config/main-local.php does not exist, creating it, please edit configuration on server!\n"
cp common/config/main-local.example common/config/main-local.php
fi
if [ ! -f common/config/params.php ]; then
printf "File common/config/params.php does not exist, creating it, please edit configuration on server!\n"
cp common/config/params.example common/config/params.php
fi
# rest init files
if [ ! -f rest/config/params.php ]; then
printf "File rest/config/params.php does not exist, creating it, please edit configuration on server!\n"
cp rest/config/params.example rest/config/params.php
fi
if [ ! -f rest/web/index.php ]; then
printf "File rest/web/index.php does not exist, creating it, please edit configuration on server!\n"
cp rest/web/index.example rest/web/index.php
fi
if [ ! -f rest/web/index-test.php ]; then
printf "File rest/web/index-test.php does not exist, creating it, please edit configuration on server!\n"
cp rest/web/index.example rest/web/index-test.php
fi
printf "\n"
printf "###############################################################################\n"
printf "# Clearing cache #\n"
printf "###############################################################################\n"
# backend runtime files
rm -rf backend/web/assets/*
rm -rf backend/runtime/debug/*
rm -rf backend/runtime/logs/*
rm -rf backend/runtime/mail/*
rm -rf backend/runtime/URI/*
rm -rf cache/*
# rest runtime files
rm -rf rest/runtime/cache/*
rm -rf backend/runtime/debug/*
rm -rf backend/runtime/logs/*
rm -rf backend/runtime/mail/*
###############################################################################
# Rebuild assets# - don't we need this???
###############################################################################
#chmod +x ../app/Console/cake
#cd ../app && Vendor/bin/cake asset_compress build --force
printf "\n"
printf "###############################################################################\n"
printf "# Composer update #\n"
printf "###############################################################################\n"
php composer.phar install
printf "\n"
printf "###############################################################################\n"
printf "# Running yii migrations #\n"
printf "###############################################################################\n"
yes | php yii migrate
printf "\n"
printf "###############################################################################\n"
printf "#Run PhpSniffer and output any errors #\n"
printf "###############################################################################\n"
php ./vendor/bin/phpcs --encoding=utf-8 --extensions=php backend --colors
php ./vendor/bin/phpcs --encoding=utf-8 --extensions=php common --colors
p
hp ./vendor/bin/phpcs --encoding=utf-8 --extensions=php rest --colors
Note: You can configure these configration file on server yourself or
replace with example files.Working like cham

Answer 2

1 / You need to access the server using ssh, then do the following:
a / gitlab-runner register
sudo gitlab-runner register
b / edit the .gitlab-ci.yml file :
stages:
- sit
deploy_linxsit:
stage: sit
variables:
APP_SIT_FOLDER: "/home/project_name"
- echo "Deploying code to SIT server"
- echo "rsync code from $CI_PROJECT_DIR to $APP_SIT_FOLDER"
# rsync only updated and new files, delete file if it's longer in repo need recursive flags
# also exluced file/dir included in excluded_list.txt
- rsync -ur --delete --exclude-from "$CI_PROJECT_DIR/excluded_list.txt" "$CI_PROJECT_DIR/" "$APP_SIT_FOLDER"
tags:
- "fill tag when register gitlab-runner"
environment:
name: "SIT"
url: http://domain/project_name
only:
# only deploy changes on branch sit, for example when merging branch branch to branch sit
- sit