I have simple groovy script that I'm using with Jenkins pipeline and fails on the git merge operation with kind of strange exception:
The script:
node("master") {
ws(env.BUILD_NUMBER.toString()) { // workspace
withCredentials([
[$class: 'UsernamePasswordBinding', credentialsId: 'bitbucket', variable: 'BITBUCKET_AUTH'],
[$class: 'UsernamePasswordBinding', credentialsId: 'bitbucket-https', variable: 'BITBUCKET_HTTPS_AUTH'],]) {
def applicationName = env.CUSTOMER_NAME
def packageName = "no.bstcm.loyaltyapp.${env.REPO_NAME}"
def googleServicesJsonContents = env.GOOGLE_SERVICES_JSON
def bitbucketRepoName = "android_loyalty_app_${env.REPO_NAME}"
def bitbucketRepoApiUrl = "https://api.bitbucket.org/2.0/repositories/boost-development/${bitbucketRepoName}"
def starterBranch = "shopping_mall"
def projectPath = "jenkins-project"
stage('Create repository on bitbucket') {
sh "curl POST -v -u $BITBUCKET_AUTH $bitbucketRepoApiUrl -H 'Content-Type: application/json' -d '{\"is_private\": true, \"project\": {\"key\": \"LOY\"}}'"
}
stage('Create local repository') {
dir(projectPath) {
sh "git init"
sh "touch README.md"
sh "git add README.md"
sh "git commit --message='Initial commit'"
}
}
stage('Merge starter') {
dir(projectPath) {
sh "git init"
sh "git remote add starter https://$BITBUCKET_HTTPS_AUTH#bitbucket.org/boost-development/app_designer_starter_android.git"
sh "git fetch starter"
sh "git checkout master" <--- FAILS HERE
sh "git remote add origin https://$BITBUCKET_HTTPS_AUTH#bitbucket.org/boost-development/$bitbucketRepoName.git"
sh "git push -u origin master"
sh "git remote remove starter"
}
}
}
}
And the exception I receive (and the pipeline broke):
[Pipeline] sh
[jenkins-project] Running shell script
+ git fetch starter
From https://bitbucket.org/***/***
* [new branch] master -> starter/master
[Pipeline] sh
[jenkins-project] Running shell script
+ git checkout master
Already on 'master'
Branch master set up to track remote branch master from starter.
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // ws
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
groovy.lang.MissingPropertyException: No such property: git for
class: org.codehaus.groovy.runtime.GStringImpl
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAd
apter.java:53)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.getProperty(ScriptBytecodeAdapter.java:458)
at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:243)
at org.kohsuke.groovy.sandbox.GroovyInterceptor.onGetProperty(GroovyInterceptor.java:52)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:308)
at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:241)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:238)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getProperty(SandboxInvoker.java:28)
at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20)
at WorkflowScript.run(WorkflowScript:36)
at ___cps.transform___(Native Method)
Do you guys have any idea what could cause this problem? I have no idea and google doesn't help much here.
trouble in this kind of groovy string:
sh ".... $bitbucketRepoName.git ...."
in this case groovy it tries to access property git of variable bitbucketRepoName
just change it this:
sh ".... ${bitbucketRepoName}.git ...."
Related
I have the following Dockerfile:
FROM ubuntu:bionic
RUN apt-get update -y && apt-get install -y nginx
EXPOSE 80
CMD ["/usr/sbin/nginx", "-g", "daemon off;"]
Building the container and running it through the following commands works:
$ docker build -t web:1.0 .
$ docker run -itd -p 8200:80 --name webserver1 web:1.0
I made an equivalent Packer file as follows:
packer {
required_plugins {
docker = {
version = ">= 1.0.1"
source = "github.com/hashicorp/docker"
}
}
}
source "docker" "ubuntu" {
image = "ubuntu:bionic"
commit = true
changes = [
"EXPOSE 80",
"CMD [\"/usr/sbin/nginx\", \"-g\", \"daemon off;\"]"
]
}
build {
name = "learn-packer"
sources = [
"source.docker.ubuntu"
]
provisioner "shell" {
inline = [
"apt-get -y update",
"apt-get install -y nginx",
]
}
post-processors {
post-processor "docker-tag" {
repository = var.docker_repo
tags = ["1.1"]
}
}
}
I'm creating the image with packer build . When I run the image through Docker
docker run -itd -p 8201:80 --name webserver2 xxx/web:1.1
I get the following error:
usr/sbin/nginx: 1: usr/sbin/nginx: Syntax error: ")" unexpected
Let's assume that we have AWS CodeBuild project which buildspec file contains the following:
"phases": {
"install": {
"runtime-versions": {
"nodejs": 14
}
},
"build": {
"commands": [
"echo Build started on `date`",
"npm ci && npm audit --audit-level=critical",
"node -r esm index.js --env=CLOUDTEST"
]
},
"post_build": {
"commands": [
"echo Build completed on `date`"
]
}
It runs successfully, but if, for example, env value is wrong, node command will fail, but the build still succeeds.
Question: what should I do to make CodeBuild project to fail when node command fails?
The build phase transitions to post_build by default, regardless whether the build succeeds or fails. To override this behaviour, explicitly set the phase's on-failure behaviour:
"build": {
"on-failure": "ABORT",
"commands": ["node BOOM"]
},
The bad node BOOM command causes the execution to fail immediately. Logs tail:
[Container] 2022/04/29 11:21:10 Command did not exit successfully node BOOM exit status 1
[Container] 2022/04/29 11:21:10 Phase complete: BUILD State: FAILED_WITH_ABORT
[Container] 2022/04/29 11:21:10 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: node BOOM. Reason: exit status 1
I have installed Pipeline Utility Steps plugin in my Jenkins and I used to use readJSON and readYaml without any issue.
A month later when I tried it i am getting the following error for both of them
groovy.lang.MissingMethodException: No signature of method: Script1.readJSON() is applicable for argument types: (java.util.LinkedHashMap) values: [[file:/data/ecsnames/dev_ECSNames.json]]
The error is similar for readYaml step as well.
I am not sure how it suddenly stopped working. Only thing I got from one of my teammates is that Jenkins was Updated to 2.235.5 version couple of weeks ago.
I used the following command
def clstrndsrvcnme = readJSON file: "/data/ecsnames/dev_ECSNames.json"
can anyone help me with this? And what does this error means?
Update*
So I was trying the above command at JenkinsURL/script. There is a little IDE to run groovy scripts. I do all kinds of Debug there.
At that location it was throwing the error.
But when I am running the same commands from a Jenkins Job, it is working perfectly fine and i am able to read values from Yaml and Json. So what I believe is that somehow JenkinsURL/script is not able to use Pipeline Utility Scripts plugin.
I am able to do my work but still wanted to understand why it is failing here JenkinsURL/script.
I spent most of a day on this same problem. This fails in the Script Console:
def jstring = '{"one":1, "two":2}'
def jobj = readJSON text: jstring
Thanks to your post, I tried it in a test pipe and it works:
pipeline {
agent any
stages {
stage('readjson') {
steps {
script {
def jstring = '{"one":1, "two":2}'
def jobj = readJSON text: jstring
echo jobj.toString()
}
}
}
}
}
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in C:\Program Files (x86)\Jenkins\workspace\readJSON
[Pipeline] {
[Pipeline] stage
[Pipeline] { (readjson)
[Pipeline] script
[Pipeline] {
[Pipeline] readJSON
[Pipeline] echo
{"one":1,"two":2}
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
I've logged an issue in the Jenkins Jira for this plugin: https://issues.jenkins.io/browse/JENKINS-65910
I'll update this post with any responses from Jenkins.
I am defining my environment variables in Jenkins-File.
I am using Pipeline Utility Steps Plugin to read a json file in the directory that has configuration.
When I echo out the read json file, the output is correct, it reads and prints the json file correctly.
When I try to access the value associated with a key in that json object, I am getting error: "No such property: internalCentralConsoleUrl for class: java.lang.String"
The json format config file looks life following:
{
"activeVersion": "19.01.303",
"internalCentralConsoleUrl": "https://11.111.111:8083/api/v1",
"dataType": "APPLICATION_JSON"
}
I am reading that file using readJSON in the pipeline.
And in the following lines, trying to access the value inside the json object using the key.
Which gives the error I mentioned above.
pipeline {
agent any
environment {
config = readJSON file: 'config.json'
baseUrl = "${config.internalCentralConsoleUrl}"
baseUrl2 = config['internalCentralConsoleUrl']
}
stages {}
}
Both the ways I tried above to read the json value are documented in the jenkins page linked here
I cannot wrap my head around what is causing an issue in this straight forward task.
Edit1: Just corrected a formatting mistake in pipeline.
I copied your example and added a stage to print the variable:
pipeline {
agent any
environment {
def config = readJSON file: 'config.json'
baseUrl = "${config.internalCentralConsoleUrl}"
}
stages {
stage('Test') {
steps {
echo baseUrl
}
}
}
}
And it prints the variable correctly without any exception:
[Pipeline] {
[Pipeline] readJSON
[Pipeline] readJSON
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] echo
https://11.111.111:8083/api/v1
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
After RNoB's comment above that it works fine in his Jenkins; I came to the conclusion that it has nothing to do with the pipeline and it might be:
a. Jenkins Plugin issue.
b. Jenkins itself.
c. Host where Jenkins is running.
So, I took the following approach:
a. I upgraded the plugins, and reinstalled them. This did not fix the problem.
b. I uninstalled Jenkins and removed all Jenkins related files and reinstalled it, and installed all the plugins again. This fixed the problem.
I still don't know what exactly was wrong, it just might be some file that was corrupt. I am not Jenkins expert but this solved the issue for me. Hope this will be helpful for somebody who is having similar issue.
I'm trying to execute an OctoberCMS migration through Jenkins with a docker command, but I have the next error in Jenkins console:
+ docker exec -u jenkins dockerfilemaster_app_1 php artisan october:up
Migrating application and plugins...
[Illuminate\Database\QueryException]
SQLSTATE[HY000] [2002] Connection refused (SQL: select * from
information_s
chema.tables where table_schema = databasename and table_name =
migrations)
[Doctrine\DBAL\Driver\PDOException]
SQLSTATE[HY000] [2002] Connection refused
[PDOException]
SQLSTATE[HY000] [2002] Connection refused
This is the declarative Jenkinsfile I'm using (the "ping" command is to ensure the database is reachable):
pipeline {
agent any
stages {
stage('Composer') {
steps {
script {
sh '''
chmod -R 777 storage
composer install
docker-compose up --build -d
'''
}
}
}
stage('Docker') {
steps {
script {
sh '''
docker exec dockerfilemaster_app_1 useradd jenkins
docker exec -u jenkins dockerfilemaster_app_1 ping -c 2 database
docker exec -u jenkins dockerfilemaster_app_1 php artisan october:up
docker exec -u jenkins dockerfilemaster_app_1 php artisan october:down --force
'''
}
}
}
}
post {
always {
sh '''
docker-compose down
'''
}
}
}
But if I connect to the Jenkins server with SSH, and I execute the same script, It works:
docker-compose up --build -d
docker exec dockerfilemaster_app_1 useradd jenkins
docker exec -u jenkins dockerfilemaster_app_1 php artisan october:up
The database config:
'mysql' => [
'driver' => 'mysql',
'host' => env('DB_HOST', 'database'),
'port' => env('DB_PORT', 3306),
'database' => env('DB_DATABASE', 'databasename'),
'username' => env('DB_USERNAME', 'root'),
'password' => env('DB_PASSWORD', 'secret'),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'prefix' => '',
],
The docker-compose.yml file:
version: '2'
services:
web:
build:
context: ./
dockerfile: web.docker
volumes:
- ./:/var/www/public
ports:
- "8081:80"
links:
- app
app:
build:
context: ./
dockerfile: app.docker
volumes:
- ./:/var/www
links:
- database
environment:
- "DB_PORT=3306"
- "DB_HOST=database"
database:
image: mysql:5.6
environment:
- "MYSQL_ROOT_PASSWORD=secret"
- "MYSQL_DATABASE=databasename"
ports:
- "3306:3306"
I can't see why If I execute the script in the server connected with SSH works, but It does not works If Jenkins execute the same script.
This is the whole Jenkins console output:
Branch indexing
> git rev-parse --is-inside-work-tree # timeout=10
Setting origin to https://gitlab.com/[...]
> git config remote.origin.url https://gitlab.com/[...] # timeout=10
Fetching origin...
Fetching upstream changes from origin
> git --version # timeout=10
> git config --get remote.origin.url # timeout=10
using GIT_ASKPASS to set credentials
> git fetch --tags --progress origin +refs/heads/*:refs/remotes/origin/*
Seen branch in repository origin/develop
Seen branch in repository origin/master
Seen 2 remote branches
Obtained Jenkinsfile from e40e597c1a6bb87221f6c0844174534b5a5a4e15
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/docker-file_master
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
[Pipeline] checkout
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://gitlab.com/[...] # timeout=10
Fetching without tags
Fetching upstream changes from https://gitlab.com/[...]
> git --version # timeout=10
using GIT_ASKPASS to set credentials
> git fetch --no-tags --progress https://gitlab.com/[...] +refs/heads/*:refs/remotes/origin/*
Checking out Revision e40e597c1a6bb87221f6c0844174534b5a5a4e15 (master)
> git config core.sparsecheckout # timeout=10
> git checkout -f e40e597c1a6bb87221f6c0844174534b5a5a4e15
Commit message: "Testing with env"
> git rev-list --no-walk 1b46891b6e4078d426f8a33dc5fc1f6b2a743f56 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Composer)
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ whoami
jenkins
+ chmod -R 777 storage
+ composer install
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Nothing to install or update
Generating autoload files
+ docker-compose up --build -d
Creating network "dockerfilemaster_default" with the default driver
Building app
Step 1/4 : FROM php:7.1-fpm
---> 894f8d826f6a
Step 2/4 : RUN apt-get update && apt-get install -y apt-utils mcrypt libmcrypt-dev mysql-client git zip unzip zlib1g-dev iputils-ping && docker-php-ext-install pdo_mysql mysqli zip
---> Using cache
---> 0881c5505ea5
Step 3/4 : COPY --from=composer:latest /usr/bin/composer /usr/bin/composer
---> Using cache
---> d1f1d74f8b0f
Step 4/4 : WORKDIR /var/www
---> Using cache
---> 60aefdca48ba
Successfully built 60aefdca48ba
Successfully tagged dockerfilemaster_app:latest
Building web
Step 1/3 : FROM nginx:1.10
---> 0346349a1a64
Step 2/3 : ADD ./vhost.conf /etc/nginx/conf.d/default.conf
---> Using cache
---> 74cadd23348a
Step 3/3 : WORKDIR /var/www
---> Using cache
---> 4142e3e3d33a
Successfully built 4142e3e3d33a
Successfully tagged dockerfilemaster_web:latest
Creating dockerfilemaster_database_1
Creating dockerfilemaster_app_1
Creating dockerfilemaster_web_1
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Docker)
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ docker exec dockerfilemaster_app_1 useradd jenkins
+ docker exec -u jenkins dockerfilemaster_app_1 whoami
jenkins
+ docker exec -u jenkins dockerfilemaster_app_1 ping -c 2 database
PING database (172.26.0.2) 56(84) bytes of data.
64 bytes from dockerfilemaster_database_1.dockerfilemaster_default (172.26.0.2): icmp_seq=1 ttl=64 time=0.158 ms
64 bytes from dockerfilemaster_database_1.dockerfilemaster_default (172.26.0.2): icmp_seq=2 ttl=64 time=0.198 ms
--- database ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.158/0.178/0.198/0.020 ms
+ docker exec -u jenkins dockerfilemaster_app_1 php artisan october:up
Migrating application and plugins...
[Illuminate\Database\QueryException]
SQLSTATE[HY000] [2002] Connection refused (SQL: select * from information_s
chema.tables where table_schema = databasename and table_name = migratio
ns)
[Doctrine\DBAL\Driver\PDOException]
SQLSTATE[HY000] [2002] Connection refused
[PDOException]
SQLSTATE[HY000] [2002] Connection refused
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Required)
Stage "Required" skipped due to earlier failure(s)
[Pipeline] parallel
[Pipeline] { (Branch: editorConfig)
[Pipeline] stage
[Pipeline] { (editorConfig)
Stage "editorConfig" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Failed in branch editorConfig
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] sh
+ docker-compose down
Stopping dockerfilemaster_web_1 ...
Stopping dockerfilemaster_app_1 ...
Stopping dockerfilemaster_database_1 ...
[3A[2K
Stopping dockerfilemaster_web_1 ... done
[3B[2A[2K
Stopping dockerfilemaster_app_1 ... done
[2B[1A[2K
Stopping dockerfilemaster_database_1 ... done
[1BRemoving dockerfilemaster_web_1 ...
Removing dockerfilemaster_app_1 ...
Removing dockerfilemaster_database_1 ...
[2A[2K
Removing dockerfilemaster_app_1 ... done
[2B[3A[2K
Removing dockerfilemaster_web_1 ... done
[3B[1A[2K
Removing dockerfilemaster_database_1 ... done
[1BRemoving network dockerfilemaster_default
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
Hmm seems when application is installing meanwhile DB instance is not able to start so may be we can add depends_on. database is always create problem in this environments.
version: '2'
services:
web:
build:
context: ./
dockerfile: web.docker
volumes:
- ./:/var/www/public
ports:
- "8081:80"
links:
- app
depends_on: <= THIS
- "app" <= N THIS
app:
build:
context: ./
dockerfile: app.docker
volumes:
- ./:/var/www
links:
- database
depends_on: <= THIS
- "database" <= N THIS
environment:
- "DB_PORT=3306"
- "DB_HOST=database"
database:
image: mysql:5.6
environment:
- "MYSQL_ROOT_PASSWORD=secret"
- "MYSQL_DATABASE=databasename"
ports:
- "3306:3306"
may be it can solve your issue. if NOT then you need to check tools suggested in reference below will help you. as they also mention DB creates problem in docker environment.
You can check reference from here : https://docs.docker.com/compose/startup-order/
if any doubt please comment.
Adding the "sleep" command worked for me. Seems that Docker is still not build when the second stage (the stage named "Docker) starts. I thought the stage starts when the previous stage It's fully finished.
This is the final Jenkins file:
pipeline {
agent any
stages {
stage('Docker') {
steps {
script {
sh '''
chmod -R 777 storage
docker-compose up --build -d
sleep 30
docker exec dockerfilemaster_app_1 useradd jenkins
docker exec dockerfilemaster_app_1 chmod -R 777 /var/www
docker exec -u jenkins dockerfilemaster_app_1 composer install
docker exec -u jenkins dockerfilemaster_app_1 php artisan october:up
docker exec -u jenkins dockerfilemaster_app_1 php artisan october:down --force
'''
}
}
}
}
post {
always {
sh '''
docker-compose down
'''
}
}
}
I put together the "docker-compose up" command with the rest of docker commands, due to putting that command in another stage did not help.