In Openshift 3.9, when I use 'oc rsync' to export jenkins data from my jenkins pod to the host's file system, I get the following error:
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1650) [generator=3.1.2]
error: exit status 24
This seems to be a known issue with the underlying linux rsync utility and has a workaround. However, because the rsync utility is called by 'oc' in my case, I cannot figure out how to deal with this issue.
Suggestions? Thanks.
Like rsync, oc rsync seems to have a --exclude option that is also described in your workaround.
So using --exclude should allow you to exclude the folder with the ephemeral files:
oc rsync --exclude='/path/to/*/tmp/' POD:/remote/dir/ ./local/dir
Related
I'm getting an error in my self-hosted github runners:
rm: cannot remove '/home/user/github-actions/1/_work/project/project/.git/index': Permission denied
It's not just this one file, but all the files that it tries to delete.
If I run something like: sudo chmod -R 777 . in the home directory, it will temporarily fix it. But the issue will come back. What can I do to fix this permenantly?
When your runner fails with this error, double-check the output of:
ls -alrth /home/user/github-actions/1/_work/project/project/.git/index
ps -eaf|grep -i runner
That way, you can see:
who has created that file
what user is executing the current runner process
That allows you to confirm a user process issue (one executing the runner, the other the files).
I wonder if Git is running as a separate user or something
No, it should not.
I have a script that runs fine when run directly from the shell of the server hosting Rundeck. It uses google-drive-ocamlfuse to mount my google drive to a local directory, creates a folder in the directory, and then unmounts.
name=New-Folder-Name
google-drive-ocamlfuse /home/user/mygoogledrive/
mkdir /home/user/mygoogledrive/$name
fusermount -u /home/user/mygoogledrive/
If I try to run this as an ad hoc command in Rundeck:
sudo ./var/lib/rundeck/scripts/create-folder.sh
... it errors out with:
Error: no DISPLAY environment variable specified
/bin/sh: 1: google-chrome: not found
/bin/sh: 1: chromium-browser: not found
/bin/sh: 1: open: not found
Cannot retrieve auth tokens.
Failure("Error opening URL:https://accounts.google.com/o/oauth2/auth?client_id=REDACTING-PERSONAL-INFO")
mkdir: cannot create directory ‘/home/user/mygoogledrive/New-Folder-Name’: No such file or directory
fusermount: failed to unmount /home/home-db/mygoogledrive: Invalid argument
I am new to Rundeck and am not yet comfortable with permissions and I don't have a good sense of how a command is being run on the server by Rundeck. It must be accessing and executing the file, given the error output, but maybe there are some limitations in the environment due to permissioning that doesn't allow for the use of certain libraries need by google-drive-ocamlfuse? Any ideas?
To use sudo on a target remote node, you need to set the sudo parameters. Otherwise, if you need to use sudo locally, the easier way is to use this plugin in your Rundeck instance.
I am running my web server on Elastic Beanstalk, and using Papertrail for logging. I am using the official .ebextensions script to get papertrail set up during deployment, but I have a problem. I use environment variables as part of my hostname used as the sender when remote_syslog uploads logs to papertrail, and while this works fine during deployment, when the 01_set_logger_hostname container command is triggered, I run into problems whenever I change environment variables by modifying the environment's configuration, since it seems an eb config call will only restart the application server, not run any of the scripts run during deployment, including the ebextensions container commands.
"/tmp/set-logger-hostname.sh":
mode: "00555"
owner: root
group: root
encoding: plain
content: |
#!/bin/bash
logger_config="/etc/log_files.yml"
appname=`{ "Ref" : "AWSEBEnvironmentName" }`
instid=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`
myhostname=${SOME_VARIABLE}_${appname}_${instid}
if [ -f $logger_config ]; then
# Sub the hostname
sed "s/hostname:.*/hostname: $myhostname/" -i $logger_config
fi
As you can see, since my hostname depends on ${SOME_VARIABLE}, I need to refresh the hostname whenever ${SOME_VARIABLE} is modified following eb config.
Is there a way to trigger a script to be run whenever an eb config command is run, so that I can not only restart my web application but also reconfigure and restart remote_syslog with the updated hostname?
This is now possible on AWS Linux 2 based environments with Configuration deployment platform hooks.
For example, you can make a shell script .platform/confighooks/predeploy/predeploy.sh that will run on all configuration changes. Make sure that you make this file executable according to git, or Elastic Beanstalk will give you a permission denied error.
Steps To Replicate
On Windows 8.
In shell (with SSH connection active):
rhc snapshot save [appname]
Error
No system SSH available. Please use the --ssh option to specify the path to your SSH executable, or install SSH.
Suggested Solution
From this post:
Usage: rhc snapshot-save <application> [--filepath FILE] [--ssh path_to_ssh_executable]
Pass '--help' to see the full list of options
Question
The path to keys on PC is:
C:\Users\[name]\.ssh
How do I define this in the rhc snaphot command?
Solution
rhc snapshot save [appname] --filepath FILE --ssh "C:\Users\[name]\.ssh"
This will show the message:
Pulling down a snapshot of application '[appname]' to FILE ...
... then after a while
Pulling down a snapshot of application '[appname]' to FILE ... DONE
Update
That saved the backup in a file called "FILE" without an extension, so I'm guessing in the future I should define the filename as something like "my_app_backup.tar.gz" ie:
rhc snapshot save [appname] --filepath "my_app_backup.tar.gz" --ssh "C:\Users\[name]\.ssh"
It will save in the repo directory, so make sure you move it out of this directory before you git add, commit, push etc, otherwise you will upload your backup too.
Basically, this is the log I get:
Started by user dontcare4free
$ hg clone --rev default ssh://hg#bitbucket.org/dontcare4free/my-repository /var/lib/jenkins/workspace/Custom-MC-Server
* failed to import extension hgext.imerge: No module named imerge
remote: Host key verification failed.
abort: no suitable response from remote hg!
ERROR: Failed to clone ssh://hg#bitbucket.org/dontcare4free/my-repository
[Custom-MC-Server] $ hg log --rev . --template {node}
java.io.IOException: Cannot run program "hg" (in directory "/var/lib/jenkins/workspace/My-Repository"): java.io.IOException: error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:475)
at hudson.Proc$LocalProc.(Proc.java:244)
at hudson.Proc$LocalProc.(Proc.java:216)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:698)
at hudson.Launcher$ProcStarter.start(Launcher.java:329)
at hudson.Launcher$ProcStarter.join(Launcher.java:336)
at hudson.plugins.mercurial.MercurialSCM.joinWithPossibleTimeout(MercurialSCM.java:299)
at hudson.plugins.mercurial.HgExe.popen(HgExe.java:191)
at hudson.plugins.mercurial.HgExe.tip(HgExe.java:171)
at >hudson.plugins.mercurial.MercurialSCM.calcRevisionsFromBuild(MercurialSCM.java:255)
at hudson.scm.SCM._calcRevisionsFromBuild(SCM.java:304)
at hudson.model.AbstractProject.calcPollingBaseline(AbstractProject.java:1205)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1194)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443)
at hudson.model.Run.run(Run.java:1376)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:175)
Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.(UNIXProcess.java:164)
at java.lang.ProcessImpl.start(ProcessImpl.java:81)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:468)
... 18 more
Jabber notifier plugin: Sending notification to: -snip-
Jabber notifier plugin: Notifying suspects
Jabber notifier plugin: Notifying culprits
Notifying upstream projects of job completion
Finished: FAILURE
As far as I can see this means that it can't find the hg executable. However, I get no such errors when I try executing hg as a build step (shell execute) with Mercurial integration disabled.
I've tried with and without changing installation directory and executable and I've even tried compiling (well, whatever of that there is) Mercurial manually from source, all to no avail.
EDIT: Silly me. I completely misread the log. The issue is not related to it not finding the hg executable at all, but it's actually because I forgot to set up my key properly.
Inspecting http://localhost:8080/systemInfo Environment Variables > PATH displays "/usr/bin:/bin:/usr/sbin:/sbin". I can't determine why this is all that Jenkins sees. When logging in as the daemon configured user, I get a much larger set of paths.
I was able to help the Mercurial Plug-in find "hg" by creating a symlink to hg.
sudo ln -s /usr/local/bin/hg /usr/bin/hg
I tried adding the following to /etc/profile (I restarted just in case)
PATH=$PATH:/usr/local/bin
export PATH
I verified that this modified my path by typing
echo $PATH
in Terminal. However, this path did not show up in Jenkins
I am able to work with the sym link solution but I'd really like to understand where Jenkins gets its Path.
Nullable is right, the issue isn't that the hg executable can't be found, but rather that the jenkins user doesn't have a public key.
The solution is as follows:
Log in as the jenkins user
Make sure the jenkins user has a public ssh key, which should be in .ssh/id_rsa.pub
If not, generate one using ssh-keygen and don't specify a passphrase
Issue cat .ssh/id_rsa.pub, copy the output.
Log into bitbucket or github, add the public key you just copied into your account
Try again!
Hope that helps, best of luck to anyone with this issue.
My first thought would be that you should check the path to the hg executable in jenkins set up, if there is such an option, it may not point to the correct path.
If that doesn't help, make sure hg is on the PATH.
Looks like there might be some set up required according to this page http://www.pixelastic.com/blog/162:failed-to-import-extension-hgext-imerge-warning-on-dreamhost
Maybe that can fix your issue?