Basically, this is the log I get:
Started by user dontcare4free
$ hg clone --rev default ssh://hg#bitbucket.org/dontcare4free/my-repository /var/lib/jenkins/workspace/Custom-MC-Server
* failed to import extension hgext.imerge: No module named imerge
remote: Host key verification failed.
abort: no suitable response from remote hg!
ERROR: Failed to clone ssh://hg#bitbucket.org/dontcare4free/my-repository
[Custom-MC-Server] $ hg log --rev . --template {node}
java.io.IOException: Cannot run program "hg" (in directory "/var/lib/jenkins/workspace/My-Repository"): java.io.IOException: error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:475)
at hudson.Proc$LocalProc.(Proc.java:244)
at hudson.Proc$LocalProc.(Proc.java:216)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:698)
at hudson.Launcher$ProcStarter.start(Launcher.java:329)
at hudson.Launcher$ProcStarter.join(Launcher.java:336)
at hudson.plugins.mercurial.MercurialSCM.joinWithPossibleTimeout(MercurialSCM.java:299)
at hudson.plugins.mercurial.HgExe.popen(HgExe.java:191)
at hudson.plugins.mercurial.HgExe.tip(HgExe.java:171)
at >hudson.plugins.mercurial.MercurialSCM.calcRevisionsFromBuild(MercurialSCM.java:255)
at hudson.scm.SCM._calcRevisionsFromBuild(SCM.java:304)
at hudson.model.AbstractProject.calcPollingBaseline(AbstractProject.java:1205)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1194)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443)
at hudson.model.Run.run(Run.java:1376)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:175)
Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.(UNIXProcess.java:164)
at java.lang.ProcessImpl.start(ProcessImpl.java:81)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:468)
... 18 more
Jabber notifier plugin: Sending notification to: -snip-
Jabber notifier plugin: Notifying suspects
Jabber notifier plugin: Notifying culprits
Notifying upstream projects of job completion
Finished: FAILURE
As far as I can see this means that it can't find the hg executable. However, I get no such errors when I try executing hg as a build step (shell execute) with Mercurial integration disabled.
I've tried with and without changing installation directory and executable and I've even tried compiling (well, whatever of that there is) Mercurial manually from source, all to no avail.
EDIT: Silly me. I completely misread the log. The issue is not related to it not finding the hg executable at all, but it's actually because I forgot to set up my key properly.
Inspecting http://localhost:8080/systemInfo Environment Variables > PATH displays "/usr/bin:/bin:/usr/sbin:/sbin". I can't determine why this is all that Jenkins sees. When logging in as the daemon configured user, I get a much larger set of paths.
I was able to help the Mercurial Plug-in find "hg" by creating a symlink to hg.
sudo ln -s /usr/local/bin/hg /usr/bin/hg
I tried adding the following to /etc/profile (I restarted just in case)
PATH=$PATH:/usr/local/bin
export PATH
I verified that this modified my path by typing
echo $PATH
in Terminal. However, this path did not show up in Jenkins
I am able to work with the sym link solution but I'd really like to understand where Jenkins gets its Path.
Nullable is right, the issue isn't that the hg executable can't be found, but rather that the jenkins user doesn't have a public key.
The solution is as follows:
Log in as the jenkins user
Make sure the jenkins user has a public ssh key, which should be in .ssh/id_rsa.pub
If not, generate one using ssh-keygen and don't specify a passphrase
Issue cat .ssh/id_rsa.pub, copy the output.
Log into bitbucket or github, add the public key you just copied into your account
Try again!
Hope that helps, best of luck to anyone with this issue.
My first thought would be that you should check the path to the hg executable in jenkins set up, if there is such an option, it may not point to the correct path.
If that doesn't help, make sure hg is on the PATH.
Looks like there might be some set up required according to this page http://www.pixelastic.com/blog/162:failed-to-import-extension-hgext-imerge-warning-on-dreamhost
Maybe that can fix your issue?
Related
I'm trying to clone a Mercurial repository hosted on BitBucket via SSH from the Windows command line.
When I add the --debug flag I can see that it never gets past the 'sending between command'
When I ctrl+c, it just shows interrupted! without any other error messages.
I'm also looking at the Resource Monitor and it's not showing any related network traffic after the first minute or so.
I have tried adding the --noupdate option and the --uncompressed option with the same results.
Windows Server 2012 with just Mercurial - No TortoiseHg
Any ideas?
Is there a reason the 'sending between command' would take awhile? The repository is about 150MB
I know I'm late to the party, since Bitbucket has long dropped the Mercurial support, still this might be relevant to those setting it up on Windows to use with any other service or in a self-hosted environment.
I assume that you've set plink as your SSH client. If not, this can be easily done by adding the ssh parameter in your Mercurial configuration. Here is what my mercurial.ini file's contents look like:
>hg config --edit
[ui]
ssh="C:\programs\putty\plink.exe" -ssh -2
I spent like two hours tinkering around with my setup before I had an Aha! moment and tried to connect to the server with plink to see if the problem lies there:
>plink.exe hg-ssh#hg.example.com -noagent id
The server's host key is not cached in the registry. You
have no guarantee that the server is the computer you
think it is.
The server's ssh-ed25519 key fingerprint is:
ssh-ed25519 255 90:8d:b0:26:c5:6e:74:2a:6c:3d:80:42:54:9e:15:47
If you trust this host, enter "y" to add the key to
PuTTY's cache and carry on connecting.
If you want to carry on connecting just once, without
adding the key to the cache, enter "n".
If you do not trust this host, press Return to abandon the
connection.
Store key in cache? (y/n)
Bingo!
As you can see here, plink prompts for the input, and we have to provide it before it can proceed any further. So this is what makes hg hang indefinitely if we don't.
You can also download Process Explorer, open it and choose Show Process Tree (Ctrl+T) from the View menu, and search for the process plink. You can see that this process is running and is a child of hg.
I answered "y", and the key was cached in a registry entry under the
Computer\HKEY_CURRENT_USER\SOFTWARE\SimonTatham\PuTTY\SshHostKeys
The new entry contains the remote server name and the key itself.
Once that was out of the way, I could go about my business using clone, push, pull, as well as other commands, like in and out:
>hg clone ssh://hg-ssh#hg.example.com/review
destination directory: review
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 4 changes to 4 files
new changesets f82779871203:e9b1a2b79f98 (1 drafts)
updating to branch default
4 files updated, 0 files merged, 0 files removed, 0 files unresolved
It looks like hg messes with the input and the output of the SSH client, and you never get to see what is going on there, nor can you interact with it. And specifying the --debug option makes no difference.
You can also supply the -sshlog option to plink to save the log protocol details of a session to a file:
[ui]
ssh="C:\programs\putty\plink.exe" -ssh -2 -sshlog plink.log
Here is a portion of the log, where it says that a host key is unknown to the client:
Incoming packet #0x2, type 21 / 0x15 (SSH2_MSG_NEWKEYS)
Event Log: Server also has ecdsa-sha2-nistp256/ssh-rsa host keys,
but we don't know any of them
Event Log: Host key fingerprint is:
Event Log: ssh-ed25519 255 90:8d:b0:26:c5:6e:74:2a:6c:3d:80:42:54:9e:15:47
And it is getting stuck around that spot waiting for the user input.
Out of curiosity I downloaded the Mercurial source code and tracked the debug messages down to the method _performhandshake in hg/mercurial/sshpeer.py:
$ hg clone https://www.mercurial-scm.org/repo/hg
$ grep -R 'sending between command' hg
$ less hg/mercurial/sshpeer.py
ui.debug(b'sending hello command\n')
ui.debug(b'sending between command\n')
Full sshpeer.py source code.
You can see up the call stack there's a call inside the instance to the _makeconnection, which in turn calls the procutil.popen4, which then uses subprocess.Popen class constructor to spawn a new process, and subprocess.PIPE is used as a value of the stdin, stdout, and stderr arguments.
And from the subprocess manual, you can see that:
PIPE indicates that a new pipe to the child should be created.
So this pretty much explains why you cannot interact with the SSH client, as all its standard input, standard output, and standard error are eaten up by Mercurial.
Anyone reading this answer, give this solution a try, and let us know if it was helpful!
I using mercurial and I am trying to point Jenkins toward custom directory. I ran into this error,
Started by user Netro
Building in workspace <path-to-workspace>
[workspace] $ hg showconfig paths.default
ERROR: Workspace reports paths.default as Not trusting file <path-to-workspace>/.hg/hgrc from untrusted user root, group root
Not trusting file <path-to-workspace>/.hg/hgrc from untrusted user root, group root
which looks different than http://ip-address/project so falling back to fresh clone rather than incremental update
ERROR: Failed to clean the repository checkout
I had searched internet and it suggested add trusted field. I had tried with following code in ~/.hgrc, /etc/mercurial/hgrc, <path-to-workspace>/.hg/hgrc
[trusted]
users = jenkins, root
But it doesn't remove error.
When I was not using custom directory option. It ran successfully.
Installation of Jenkins and repository was done with user root on Ubuntu 1204.
Any suggestions will be helpful.
It seems, hg was not able to accept jenkins as owner of the projects. Jenkins in default directory creates workspace with user Jenkins. So it is able to run it. I added Jenkins in group root and changed the owner of project from root to jenkins. Now projects are updated from Jekins.
I've got my code in a Mercurial repository (secured with a self-signed certificate) and I'm trying to set up Jenkins to work with it.
I've got the Mercurial plugin installed in Jenkins (pointing to an install of TortoiseHg on the Jenkins Server/Slaves) and the Jenkins Job is properly configured to grab the source from the repository.
When I build manually (ie, via the web interface) everything works as expected.
However, it seems like the polling of the repository does not succeed as I get output similar to the following:
Started on Apr 27, 2012 1:07:41 PM
[<jobname>] $ hg pull --rev default
warning: <MercurialServerIP> certificate with fingerprint e3:5f:5e:ea:4f:da:ef:a4:0b:4a:bb:00:e8:31:59:de:ce:d0:28:94 not verified (check hostfingerprints or web.cacerts config setting)
abort: mercurial_keyring: http authorization required but program used in non-interactive mode
[<jobname>] $ hg log --style <workspace>\<jobname>\tmp688470509422797505style --branch default --no-merges --prune 65d180b20a1e625841c8385709c86b83c3e10421
Done. Took 1.9 sec
No changes
I've previously done a manual clone of a repository so that I was able to enter the user's password to work with the Mercurial keyring extension for the authorization, but based on the error output it doesn't seem as though that's being applied.
How can I configure Jenkins or the machine running the build to do the polling successfully?
This may not be the best way to address the issue, but it worked for me and I'm able to move on.
The only way I was able to figure out how to get the server to remember the password in my setup was to specify it manually in \mercurial.ini .
NOTE: You may also have to remove the mercurial_keyring line from mercurial.ini. (This disables the keyring extension since we're specifying everything manually.)
I had previously believed that cloning a repository once on the server would let it remember the password, but this doesn't seem to work with the polling functionality in Jenkins (although it did work with my actual build scripts when they were executed on the server).
I'm not particularly pleased with having the password in plain text on the server, but until I find a better way to get the polling to work I can live with this.
Using the "kilnauth extension" you can have you credentials stored on your machine. This way you don't have to configure anything special on Jenkins.
$ hg help kilnauth
kilnauth extension - stores authentication cookies for HTTP
repositories. This extension knows how to capture Kiln
authentication tokens when pushing over HTTP.
This means you only need to enter your login and password once;
after that, the FogBugz token will be stored in your home
directory, allowing pushing without a password.
For instructions on how to install it follow: http://kiln.stackexchange.com/questions/341/how-can-i-install-kilns-mercurial-extensions-manually
After installing and playing around with mercurial , I am trying to get Hudson to clone the repository so it can build my project.
At the moment the following task works.
I Can sync to my external host and the code shows up on that host.
Now I am trying to configure hudson / jenkins to access the code on my host.
But unfortunately I am rolling on a error:
Started by user anonymous
$ hg clone --rev default https://bitbucket.org/*/testproject "F:\Hudson\jobs\testproject\workspace"
abort: http authorization required
ERROR: Failed to clone https://bitbucket.org/*/testproject
[workspace] $ hg log --rev . --template {node}
java.io.IOException: Cannot run program "hg" (in directory "F:\Hudson\jobs\testproject\workspace"): CreateProcess error=267, The directory name is invalid
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at hudson.Proc$LocalProc.<init>(Proc.java:244)
at hudson.Proc$LocalProc.<init>(Proc.java:216)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:698)
at hudson.Launcher$ProcStarter.start(Launcher.java:329)
at hudson.Launcher$ProcStarter.join(Launcher.java:336)
at hudson.plugins.mercurial.MercurialSCM.joinWithPossibleTimeout(MercurialSCM.java:298)
at hudson.plugins.mercurial.HgExe.popen(HgExe.java:191)
at hudson.plugins.mercurial.HgExe.tip(HgExe.java:171)
at hudson.plugins.mercurial.MercurialSCM.calcRevisionsFromBuild(MercurialSCM.java:254)
at hudson.scm.SCM._calcRevisionsFromBuild(SCM.java:304)
at hudson.model.AbstractProject.calcPollingBaseline(AbstractProject.java:1186)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1175)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:523)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:418)
at hudson.model.Run.run(Run.java:1362)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:145)
Caused by: java.io.IOException: CreateProcess error=267, The directory name is invalid
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(ProcessImpl.java:81)
at java.lang.ProcessImpl.start(ProcessImpl.java:30)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 18 more
Finished: FAILURE
What actions do i need to do to tell Hudson to use username x and password y to acces the data?
Edited => Found how to integrate ssh .
Used SSH instead of https
Download putty.exe, puttygen.exe, pageant.exe, and plink.exe from the PuTTY website.
Start puttygen and generate a key in OPENSSH FORMAT (hudsons format) (=> How to use Svn + SSH )
Click the Save private key button and save the .PPK file somewhere.
Click the Save public key button and save it.
Go to your website and enter the public ssh-key
Run pageant.exe. The pageant icon (a computer wearing a hat) will show up in the status tray.
Right-click the pageant icon and choose Add Key.
Choose the .PPK file you saved earlier and type in its passphrase.
The following (end part is copied) from Ted Naleid (Thank you!) blog witch can be found here : Hooking up hudson to your ...
Install the Mercurial plugin in Hudson
All that’s left to do now is install
the Mercurial plugin in hudson. In a
browser, go to
http://INSERT_YOUR_IP_HERE:8080.
Hudson should come up.
Click on “Manage Hudson” and go to
“Manage Plugins”. Go to the
“Available” tab, check “Hudson
Mercurial plugin” and hit the
“Install” button. Hudson will prompt
you to restart, and then it’s
installed.
After that, just create a new job and
you’ll have a new “mercurial” option
in the “source control management”
section. Select that and put the ssh
URL in the “Repository URL” field.
Then put “default” in the “branch”
field and set up the rest of the job
to build/test your code (an exercise
left to the reader).
and here it is the first succesfull build !
Conclusion : This is a summary of all the small blogpost scattered arround the internet. I hope this post helps you in starting hudson and mercurial.
I think the problem is not related to username and password. Your stacktrace tells you there's something wrong with the path F:\Hudson\jobs\testproject\workspace.
Cannot run program hg (in directory
"F:\Hudson\jobs\testproject\workspace")
The directory name is invalid
Anyway, you can specify the username and password in the URL like: http://user:password#mydomain.org.
To authenticate the Jenkins/Hudson Mercurial plugin with BitBucket I too found it useful to use the SSH protocol instead of HTTPS particularly since:
there doesn't seem to be a way to store your HTTPS credentials to BitBucket with the Mercurial Jenkins plugin, but with SSH you can safely and securely store your credentials
with SSH you can configure it to use compression, which Mercurial doesn't do natively.
Good instructions for setting up SSH access to BitBucket are available here: http://confluence.atlassian.com/display/BITBUCKET/Using+SSH+to+Access+your+Bitbucket+Repository
Notes:
If you are running Jenkins/Hudson on a *nix server, you will want to login as the user running the Jenkins process and perform these operations from that users home directory, so the configurations will be found by that user (e.g. on my Debian server installation of Jenkins standalone, the user 'jenkins' is created and the home directory is set to '/var/lib/jenkins' [not /home/jenkins] - where I performed the instructions provided at the above link).
I found it very helpful to assure the hg clone command worked from the command line before attempting to have Jenkins call it.
IMPORTANT: In order to get this to work, I had to generate a key ** without ** a passphrase.
You can add the following lines to jenkins .hgrc file (usually /var/lib/jenkins/.hgrc)
[auth]
bitbucket.prefix = https://bitbucket.org/your_user/...
bitbucket.username = your_user
bitbucket.password = ******
See http://www.selenic.com/mercurial/hgrc.5.html#auth
You can add your scm credentials in the 'Credentials` section of Jenkins:
Also change the job configuration to use the credentials:
This is probably a simple problem and I'm feeling exceptionally dumb because I can't find a any kind of documentation.
I've just installed TeamCity 5 and I want to get files from my Mercurial hosting and there is two fields I just can't figure out.
HG Command path. What should I put here? The path to a file containing what? Can I get an example of that file somewhere?
The host is using Mercurial over SSH where do I define my private key?
Pull changes from? Should I put the address I'm cloning from i.e. ssh://username#myhost.something/project
I figured this out for my TeamCity 5 server last week.
HG Command path: HG
Pull changes from: https://bitbucket.org/.../.../
Don't put the username# in the URL. This is specificed as in the Username/Password fields. If you include the username in the URL it'll fail as there is a bug in the configuration tool. You'll also see a screenshot of the configuration attached to the thread:
http://www.jetbrains.net/devnet/message/5254640#5254640
I'd suggest getting things working with HTTPS and then moving to SSH if possible. This breaks things down into two easier to solve configuration problems. I used the following tutorial to get SSH going on my Windows client machine.
http://www.codza.com/mercurial-with-ssh-setup-on-windows
I've not set this up on my TeamCity server yet. However I did get TeamCity to pick up my Mercurial.ini settings by putting the ini file in \Documents and Settings\TeamCity, which is the account the service runs under.
I've not used team city, but I think hg command path is probably the full path to your local mercurial executable. For me (on linux) that's:
$ type hg
hg is /usr/bin/hg
On windows it's where the 'hg' executable in your system path was placed by whichever (of the many) windows installers for mercurial you used.
Pull changes from sounds like the URL to the repo, so:
ssh://username#myhost.something/project
or
ssh://username#myhost.something//project # note the _two_ double slashes
if you're using absolute paths on the server side.
Your private key location/specification depends on what you're using for ssh and whether or not you're running ssh-agent, but here's a links that explicitly points from within mercurial.ini, which seems sound:
http://dev.openttdcoop.org/projects/home/wiki/Configuring_TortoiseHg_(Windows)#Pointing-to-you-Private-key