Summary
I've got a build in TeamCity using Mercurial as the VCS and it's repeatedly failing for one of these two reasons:
hg init - repository already exists, except I deleted the whole directory before this so it definitely didn't exist.
hg pull - timed out waiting for lock, but the lock it's waiting for seems to be its own lock.
I'm really hoping that someone has come across this before, or might be able to give me some ideas for how to troubleshoot it anyway.
Setup
I'm using TortoiseHg as the mercurial client, and I've updated it
(and hence Mercurial) to version 4.6.1 on both the build server and
agent.
The agent is running on a Windows 7 VM.
I have a Windows 10 VM with the same TeamCity/Mercurial setup that's
working fine.
The repo being pulled from is located on a network share.
The folder being pulled to is on a secondary drive on the VM.
The two problems I'm seeing are as follows:
1. Hg init failure
Steps:
Manually delete the whole working directory from the buildagent, so that's .hg folder and it's parent folder.
The working folder doesn't even exist now, so TeamCity will have to completely recreate the folder.
Run build on TeamCity, with clean all files selected.
Build starts, creates directory and calls hg init.
Error message that hg init failed because the "repository already exists".
When I look at the directory I can see a .hg folder, and some files inside it including a wlock file.
2. Pull failure
Steps:
Leave the working directory from problem 1 in place, including the .hg directory.
Ensure any lock files are deleted and hg recover has been run just in case.
Run build on TeamCity, without cleaning the directory.
The logs show hg pull starting and bundling files, but also says "waiting for lock on working directory of E:\blah held by process '3408' on host 'BUILDAGENT'
3408 here is an example, the number changes every time and corresponds to the hg.exe process that seems to be doing the pull.
Eventually after a lot of bundling and files messages I'll get a message saying it timed out waiting for the lock.
But of course the lock it's waiting for seems to be the lock it's holding itself!
If I delete the wlock file during this time, I'll see messages saying "got lock after X seconds" and immediately after it "waiting for lock on repository E:\blah held by process '3408' on host 'BUILDAGENT'. Then eventually it'll fail with a message about an abandoned transaction.
Does anyone have any ideas?
I have a repo on a network drive (served by Windows server), with local repos pushing/pulling to it on the various machines I work on.
I just dealt with this problem, and solved it by cloning the repo from the network drive to a local disk, pushing, then cloning it back again. The machine from which I did this had not problem pushing further changes after this.
Now I just tried pushing from my laptop, and this happens:
% hg --debug push "Z:\[main repo]"
pushing to Z:\[main repo]
query 1; heads
searching for changes
all remote heads known locally
listing keys for "bookmarks"
2 changesets found
list of changesets:
2ed25c8975482734e3b9eed828573fd711d26fd8
19a424c011ffd0c887cf1d54ed0b537a6c1af714
adding changesets
add changeset 2ed25c897548
add changeset 19a424c011ff
adding manifests
adding file changes
adding GEM.py revisions
transaction abort!
rollback completed
abort: No usable temporary file name found
[command returned code 255 Thu Mar 09 18:51:11 2017]
The only info pertaining to this error message I have found so far is this, and I definitely have no files named con.*in my project. There are several named con*.py but they have never been a problem, and both the laptop and my workstation are running Windows 7, and I've been working on this project for a few years now.
I have happily pushed from this laptop for over a year, and it was never a problem. I don't really have any good idea where to even start looking. Could it be connected to the fact that my workstation had the main repo opened at the same time? It was definitely not doing anything to it at the time.
Update:
I ran hg verify, and this is what it returns -- no problem as far as aI can tell
% hg --debug verify
repository uses revlog format 1
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
73 files, 74 changesets, 226 total revisions
[command completed successfully Fri Mar 10 08:58:02 2017]
I had faced the same error as well.
I just ran tortoise hg as as administrator and that fixed it for me
I don't have an answer yet but I would try the following:
Update to the latest mercurial version (4.1) and try again
Verify the repo integrity with hg verify
Although I understand it always worked as is, try to rename all the con.py files. The thing with CON is that it represents a device, I think it comes from DOS times :-)
If I understand correctly, you push to Z:[main repo] where Z: is a Windows share. Try to push to the same repo in another way, with ssh (requires some setup, yes)
Good luck, very bizarre problem :-/
I am on ubuntu 12.04 and using hg 2.5.2
I did a hg init and am trying to clone a repository using hg clone, hg dies with the following error
transaction abort!
rollback completed
abort: integrity check failed on 00changelog.i:16630!
I am pretty sure the repository isn't corrupted because I cloned the same repository on three or four other machines, this is specific to my machine.
Sometimes it fails with different errors (corrupted MAC on input) or at different points integrity check failed on 00changelog.i:11090!
Please help. Thanks
Try to run hg verify on remote repository and on your local copy. What does it say?
Update your Mercurial to the newest version.
Do you clone on hard drive? Then try to clone on another hard drive or on USB drive. And vice versa: if you clone on USB drive then try to clone on hard drive. Bad storage devices are a common reason of repository corruption as people say in comments to this answer.
Looks like there is some optimization that is turned on by default in Ubuntu 12.04 onwards.
This fixed the problem
ethtool -K eth0 rx off tx off
I've been using Mercurial v 1.1 for several months to version documents and other files. Yesterday it suddenly failed with the message:
waiting for lock on working directory
This happens in all projects I have under .hg control. Mercurial also thinks that all files in all projects have changed.
There is no .hg/store/lock file in the project it says it is waiting on the lock for.
The only thing that could have caused this is that Windows installed security patch on my computer overnight.
Has anyone else seen this with Mercurial?
I've had success by deleting that file .hg/wlock entirely if it exists, then everything is back to normal. If you are worried about losing something, just make a copy
For working directory, the lock is .hg/wlock. Does the file exists?
For rebuilding the dirstate (beware it won't restore changes like adds/remove/renames/copies), you can use hg debugrebuildstate.
I upgraded to hg version 1.3.1 and everything works now.
I must have had corruption in the 1.1.1 binaries (from Cygwin).
Cygwin is still on 1.1.
To find out which file is locking the directory, in your working directory:
hg debuglocks
This should give a result indicating which file is locking the directory e.g.
lock: free
wlock: (461232s)
To unlock use force:
hg debuglocks --force-wlock
or:
hg debuglocks --force-lock
for more information:
hg debuglocks -h
Note this paragraph:
Locks protect the integrity of Mercurial's data, so should be treated
with care. System crashes or other interruptions may cause locks to
not be properly released, though Mercurial will usually detect and
remove such stale locks automatically.
Got a bluescreen in windows while cloning a mercurial repository.
After reboot, I now get this message for almost all hg commands:
c:\src\>hg commit
waiting for lock on repository c:\src\McVrsServer held by '\x00\x00\x00\x00\x00\
x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
interrupted!
Google is no help.
Any tips?
When "waiting for lock on repository", delete the repository file: .hg/wlock (or it may be in .hg/store/lock)
When deleting the lock file, you must make sure nothing else is accessing the repository. (If the lock is a string of zeros or blank, this is almost certainly true).
When waiting for lock on working directory, delete .hg/wlock.
I had this problem with no detectable lock files. I found the solution here: http://schooner.uwaterloo.ca/twiki/bin/view/MAG/HgLockError
Here is a transcript from Tortoise Hg Workbench console
% hg debuglocks
lock: user None, process 7168, host HPv32 (114213199s)
wlock: free
[command returned code 1 Sat Jan 07 18:00:18 2017]
% hg debuglocks --force-lock
[command completed successfully Sat Jan 07 18:03:15 2017]
cmdserver: Process crashed
PaniniDev% hg debuglocks
% hg debuglocks
lock: free
wlock: free
[command completed successfully Sat Jan 07 18:03:30 2017]
After this the aborted pull ran sucessfully.
The lock had been set more than 2 years ago, by a process on a machine that is no longer on the LAN. Shame on the hg developers for a) not documenting locks adequately; b) not timestamping them for automatic removal when they get stale.
Coworker had this exact problem today, after a BSoD while trying to push. He had to:
delete the file .hg/store/lock (as per the accepted answer)
delete the file .hg/store/phaseroots (as per this TortoiseHG bug report)
Then his repo worked again.
EDIT: As per #Marmoute's comment - when dealing with lock-related issues, using hg debuglock is a safer alternative to blindly deleting the .hg/store/lock file.
I am very familiar with Mercurial's locking code (as of 1.9.1). The above advice is good, but I'd add that:
I've seen this in the wild, but rarely, and only on Windows machines.
Deleting lock files is the easiest fix, BUT you have to make sure nothing else is accessing the repository. (If the lock is a string of zeros, this is almost certainly true).
(For the curious: I haven't yet been able to catch the cause of this problem, but suspect it's either an older version of Mercurial accessing the repository or a problem in Python's socket.gethostname() call on certain versions of Windows.)
I had the same problem. Got the following message when I tried to commit:
waiting for lock on working directory of <MyProject> held by '...'
hg debuglock showed this:
lock: free
wlock: (66722s)
So I did the following command, and that fixed the problem for me:
hg debuglocks -W
Using Win7 and TortoiseHg 4.8.7.
I had the same problem on Win 7.
The solution was to remove following files:
.hg/store/phaseroots
.hg/wlock
As for .hg/store/lock - there was no such file.
I do not expect this to be a winning answer, but it is a fairly unusual situation.
Mentioning in case someone other than me runs into it.
Today I got the "waiting for lock on repository" on an hg push command.
When I killed the hung hg command I could see no .hg/store/lock
When I looked for .hg/store/lock while the command was hung, it existed. But the lockfile was deleted when the hg command was killed.
When I went to the target of the push, and executed hg pull, no problem.
Eventually I realized that the process ID on the hg push was lock waiting message was changing each time. It turns out that the "hg push" was hanging waiting for a lock held by itself (or possibly a subprocess, I did not investigate further).
It turns out that the two workspaces, let's call them A and B, had .hg trees shared by symlink:
A/.hg --symlinked-to--> B/.hg
This is NOT a good thing to do with Mercurial. Mercurial does not understand the concept of two workspaces sharing the same repository. I do understand, however, how somebody coming to Mercurial from another VCS might want this (Perforce does, although not a DVCS; the Bazaar DVCS reportedly can do so). I am surprised that a symlinked REP-ROOT/.hg works at all, although it seems to except for this push.
If the locked repo was the original, I can't imagine it was modifying it to clone it, so it was only preventing you from changing it in the middle and messing up the clone. It should be fine after removing the lock.
The new cloned copy (if it was a local clone) could be in any sort of malformed state, though, so you should throw it out and start it over. (If it was a remote clone, I would hope it failed and already threw out the incomplete copy.)
I encountered this problem on Mac OS X 10.7.5 and Mercurial 2.6.2 when trying to push. After upgrading to Mercurial 3.2.1, I got "no changes found" instead of "waiting for lock on repository". I found out that somehow the default path had gotten set to point to the same repository, so it's not too surprising that Mercurial would get confused.
If it only happens on mapped drives it might be bug https://bitbucket.org/tortoisehg/thg/issue/889/cant-commit-file-over-network-share. Using UNC path instead of drive letter seems to sidestep the issue.