Openshift - Execution of post execute step failed - openshift

I have node.js app and during the deployment after installing dependencies the following error had occured:
error: Execution of post execute step failed
warning: Failed to remove container "a167df5e218c392e42ec772d5c22311f88043ff99c71ce1a08e7af535ac3817b": Error response from daemon: {"message":"Driver devicemapper failed to remove root filesystem a167df5e218c392e42ec772d5c22311f88043ff99c71ce1a08e7af535ac3817b: Device is Busy"}
error: build error: building my-pokus/hello-seattle-2:d4b8ecde failed when committing the image due to error: Cannot connect to the Docker daemon. Is the docker daemon running on this host?

So that happens when node that the build is scheduled on, is being shutdown or restarted. Try to re-spin the build so the scheduler will, put it on an available node.
Should work :)

The problem is that your image is too big, if the commit is longer than 2mins, this error happens.
I found a workaround here : github origin 13515
Shrink your Docker Image :)
Use a more recent S2I-Builder:
In order to temporarily use another version of the docker image, the
easiest way seems to be to simply pull the new image and tag it as the
one used by openshift:
docker pull docker.io/openshift/origin-sti-builder:v1.5.0-rc.0
docker tag docker.io/openshift/origin-sti-builder:v1.5.0-rc.0 docker.io/openshift/origin-sti-builder:v1.4.1

Related

Getting logs/more information during start-build command execution

Jenkins pipeline is building Docker images. OpenShift plugin(s) are used for the same.
An example command:
openshift.selector(BUILD_CONFIG_NAME, "${appBcName}").startBuild("--from-dir=${artifactPath}", '--wait','--follow')
While this works smoothly most of the time, whenever this command fails due to some underlying platform issues, almost no information is seen in the Jenkins build job console:
[Pipeline] }
[start-build:buildconfig/amld5-car-reporting-spacetime-ubi-openshift-java-runtimejd] ............................................................
[start-build:buildconfig/amld5-car-reporting-spacetime-ubi-openshift-java-runtimejd] Uploading finished
[start-build:buildconfig/amld5-car-reporting-spacetime-ubi-openshift-java-runtimejd] Error from server (BadRequest): unable to wait for build amld5-car-reporting-spacetime-ubi-openshift-java-runtimejd-857 to run: timed out waiting for the condition
[Pipeline] }
ERROR: Error running start-build on at least one item: [buildconfig/amld5-car-reporting-spacetime-ubi-openshift-java-runtimejd];
{err=, verb=start-build, cmd=oc --server=https://api.scp-west-zone02-z01.net:6443 --certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt --namespace=sb-1166-amld5-car-service-se --token=XXXXX start-build buildconfig/amld5-car-reporting-spacetime-ubi-openshift-java-runtimejd --from-dir=./build/libs --wait --follow -o=name , out=Uploading directory "build/libs" as binary input for the build ...
............................................................
Uploading finished
Error from server (BadRequest): unable to wait for build amld5-car-reporting-spacetime-ubi-openshift-java-runtimejd-857 to run: timed out waiting for the condition
, status=1}
[Pipeline] // catchError
I need more verbosity, detailed error information. I checked the start-build command reference, and I thought --build-loglevel [0-5] might help here. When I used it, I got a warning that since I am using source type as 'Binary' in the BuildConfig, logging isn't supported(seriously???)
NOTE: the selector returned when -F/--follow is supplied to startBuild() will be inoperative for the various selector operations.
Consider removing those options from startBuild and using the logs() command to follow the build output.
[start-build:buildconfig/casc-docs-spacetime-ubi-openshift-java-runtimeadoptopenjdk] WARNING: Specifying --build-loglevel with binary builds is not supported.
[start-build:buildconfig/casc-docs-spacetime-ubi-openshift-java-runtimeadoptopenjdk] WARNING: Specifying environment variables with binary builds is not supported.
[start-build:buildconfig/casc-docs-spacetime-ubi-openshift-java-runtimeadoptopenjdk] Uploading directory "build/libs" as binary input for the build ...
[start-build:buildconfig/casc-docs-spacetime-ubi-openshift-java-runtimeadoptopenjdk] ..
How do I get more logs, info. while executing the start-build command?
I was facing the same problem, I just used something like:
def build = openshift.selector(BUILD_CONFIG_NAME, "${appBcName}").startBuild("--from-dir=${artifactPath}", '--wait','--follow')
build.logs('-f')
And so far it seems to work, I got the logs from my openshift build in my Jenkins pipeline. Now I'll try to get the logs only if build does not Complete, to reduce the overall logs.
(for future searchers like me ^^)

Error while running "podman run"; error adding pod to CNI network "podman": unexpected end of JSON input

I'm new to podman, and I just trying to run containers on it.
(podman version 3.4.0, installed by brew, intel Core MAC)
However, when I trying to run "podman run {image-name}", below errors were thrown.
$ podman run -ti -d --name web httpd 125
Error: error configuring network namespace for container b0e70d672cb66005833c0a300c8661b88eab49e942c240d69d17587e0b75c47b: error adding pod web_2_web_2 to CNI network "podman": unexpected end of JSON input
$ podman run centos:7
Error: error preparing container a6d0bc1ad217cd8207935561dc8ff7bd33672da3fa513917f9965cb39520c449 for attach: error configuring network namespace for container a6d0bc1ad217cd8207935561dc8ff7bd33672da3fa513917f9965cb39520c449: error adding pod quirky_snyder_quirky_snyder to CNI network "podman": unexpected end of JSON input
By reading https://issueexplorer.com/issue/containers/podman/11452, I removed ~/.docker/, but the solution doesn't work in my case.
Of course, the error message says there was "unexpected end of JSON input", but I don't know how to fix it. Could anyone guess why podman didn't work even running these base images, or how to debug it?
Thanks in advance.
on macos, current machine version 3.3.1 has this problem. I had this problem on server version 3.3.1 and I do not encounter it on server version 3.4.0. You can check server version with podman version.
Try removing current machine and installing a newer one
podman machine stop
podman machine rm
podman machine init --image-path next
podman machine start
Check server version again with podman version.
Try running your image again.

apache drill on cluster start error

I install apache drill on a cluster with 3 nodes.
When I use the following command to start it,it will not really running.
bin/drillbit.sh start
error
I don't know how to solve it and want you help.
The zookeeper is running without problems.
Then I check the log, and it show the following infos:
Exception in thread "main" org.apache.drill.exec.exception.DrillbitStartupException: Failure while initializing values in Drillbit.
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:287)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271)
at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267)
Caused by: org.apache.drill.exec.exception.DrillbitStartupException: Problem in finding the native library of JPAM (Pluggable Authenticator Module API). Make sure to set Drillbit JVM option 'java.library.path' to point to the directory where the native JPAM exists.:no jpam in java.library.path
I check the java.library.path, it is the following:
/home/hadoop/bigdata/hadoop-2.7.2/lib/native/::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
So, I add the following setting:
declare -x DRILL_JAVA_LIB_PATH="/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib"
However, it not work and turn out the same problem like before.
The declare -x DRILL_JAVA_LIB_PATH snippet you provided will not point drill to the pam library. Please follow all the instructions in the Drill docs here https://drill.apache.org/docs/using-jpam-as-the-pam-authenticator/
Note: you will have to perform those steps on all 3 nodes of your cluster.

"init: Failed to spawn google main process: unable to execute: No such file or directory" in /var/log/syslog

When booting up or restarting one of my instances in Google Compute Engine, I'm getting this error in /var/log/syslog.
Without the "google" service/process, I'm unable (among other things) to update my SSH keys from GCE's Metadata.
Running
sudo service google start
gives me the error
start: Job failed to start
with the same error line in /var/log/syslog.
Any ideas on how to resolve this issue?
Fixed myself by following the "From Source Repository" installation instructions in this git repo: https://github.com/GoogleCloudPlatform/compute-image-packages
It appears that all the /usr/... and /etc/... files were missing.

How to debug the cartridge scripts in Openshift?

I want to develop a new cartridge for my own use. I use OpenShift Cartridge Development Kit to start my work. My building script is written in .openshift/action_hooks/build and it can be successfully executed.
But when I tried to use the command displayed on the homepage of the CDK project - "rhc create-app mynewcart http://##YOUR-DOMAIN##/manifest/##YOUR-COMMIT-ID##" - to create an app, I got the following error: "Unable to complete the requested operation due to: An invalid exit code (1) was returned from the server ex-std-node161.prod.rhcloud.com. This indicates an unexpected problem during the
execution of your request."
How can I trace the progress of deploying and find out where is the code that caused the problem? Is there any log file available for me to analyse?
Use the --debug option when running the command and you'll get much more detailed output:
rhc create-app mynewcart http://##YOUR-DOMAIN##/manifest/##YOUR-COMMIT-ID## --debug