I'm getting daily repository corruption on my mercurial.
Every time it says:
abort: integrity check failed on data/*classified*:16!
i ran hg verify and it said:
*classified* files, *classified* changesets, *classified* total revisions
16 integrity errors encountered!
(first damaged changeset appears to be 9259)
EDIT: The supposed corruption above turned out to be some changes i did in the files, but it still is actually getting corrupted from time to time so that when i pull it says the repository is corrupted.
Found the problem. A script i was running was modifying the .hg files.
Related
Due to unknown reasons an HG command failed with the message:
abort: abandoned transaction found - run hg recover!
But when I tried to use recover to get rid of the abandoned transaction, I got a different error:
$ hg recover
rolling back interrupted transaction
attempted to truncate path/to/file to 12345 bytes, but it was already 456 bytes
and it aborted. The actual file was named like:
_some_filename.cs.i
which is an internal HG data file. So it seems like the HG data records for some_filename.cs are badly clobbered. And indeed running hg verify shows errors like this:
# hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
warning: revlog 'data/project/folder/some_filename.cs.i' not in fncache!
13691: empty or missing project/folder/some_filename.cs
project/folder/some_filename.cs#13691: manifest refers to unknown revision b269f6036278
project/folder/some_filename.cs#13741: manifest refers to unknown revision 651b96abf6da
...
(goes on for a long time)
which corroborates that the file is damaged but doesn't do anything useful to fix it.
hg recover --help doesn't show anything else that I can do...
And aside from the apparent damage to this one file, no ordinary HG commands work at this point. All of them report the repository is damaged. How can I recover from this?
I concluded that this was just a situation that HG was not designed to deal with. For whatever reason HG's internal data file (the .i file) was destroyed.
It was helpful to review the HG source code which produced the key error:
if fp.tell() < o:
raise error.Abort(
_(
b"attempted to truncate %s to %d bytes, but it was "
b"already %d bytes\n"
)
% (f, o, fp.tell())
)
(https://fossies.org/linux/mercurial/mercurial/transaction.py)
I'm not especially familiar with HG internals but this seemed to make it clear enough that HG was not able to handle the situation (understandably - arbitrary destruction of one of its data files leaves few options!).
The best workaround I could come up with was to manually copy the damaged .i file from another copy of the repository (on another PC). I didn't actually have local changes to that source file, so this seemed reasonable.
I copied the file (replacing the damaged original having made a backup first).
Then ran hg recover. This was now able to resolve the original "abandoned transaction" issue. Other HG commands work as well.
Worth noting that running hg verify still reports some errors (though many fewer). Perhaps the history of this single file is still not right; ultimately I think I will need to re-clone this repository but at least I can complete my immediate tasks without losing any work.
Recently migrated to Mercurial.
Due to heavy use of externals in old SVN repo we are using Subrepos accordingly and have a CI server that does pulls / pushes to central repositories often. So it's a bit hard to trace what exactly happened and developers can't reproduce the exact steps.
But, after pulling we got errors likes this:
RepoLookupError: unknown revision '766981bc81dc78fe24d5fe5c7d68e36c66858e73'
abort: unknown revision '766981bc81dc78fe24d5fe5c7d68e36c66858e73'!
And such changesets could not be found anywhere, nor on server nor in local repositories. Got this situation twice per day.
Somehow, from the server comes a .hgsubstate that refers to unknown subrepository changeset.
And we didn't do anything potentially harmful, just usual commits / pulls / merges.
As of our understanding - this is an impossible situation (you can't commit a .hgsubstate referring to uncommitted or not existing subrepository changeset).
Any ideas what we could be doing wrong or how this could happen?
edit:not using mq either
To create an unknown (invisible) revision, do like this.
Client 1
Commmit a change in a subrepo
Link the commit in the subrepo with a commit in the subrepo (the subrepo changeset id will be updated in the .hgsubstate file)
Rollback the subrepo commit (now you got yourself a link in the master repo to a non-existing changeset in the subrepo)
Push both repositories to the server
Client 2
Pull from the server
Error occur!
This is what i get when i do hg verify :
repository uses revlog format 1
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
includes/base/class/ViewInstanceAdapter.class.php#7: broken revlog! (index data/includes/base/class/ViewInstanceAdapter.class.php.i is corrupted)
warning: orphan revlog 'data/includes/base/class/ViewInstanceAdapter.class.php.i'
158 files, 61 changesets, 270 total revisions
1 warnings encountered!
1 integrity errors encountered!
(first damaged changeset appears to be 7)
I do not use Mercurial for a long time and i don't understand what this means.
(I'm on windows using TortoiseHg, and the project is local only)
As said before (although you already confirmed this doesn’t work), you should start by trying to clone the repository; if the problems are related to the dirstate this can bypass it.
Next, every clone contains a complete repository, so every clone is effectively a back-up. Don’t you have a central server or colleague or another local copy? Try cloning that, then pulling from your corrupted repository. As the first damaged changeset is reported as being no. 7 (out of 270), this should be a pretty old one so likely easy to recover, and hopefully the damage does not prevent Mercurial from pulling changesets beyond that.
A third option you could try is to run a Mercurial-Mercurial conversion on your repository (hg convert repo repo-copy); a verbatim conversion should the keep changeset IDs intact, although it will probably run into the same problem. You could also try to specify a filemap to filter out the ViewInstanceAdapter file.
Because the damaged changeset is so old, and given that Mercurial uses an append-only writing method, the probable cause for this problem is a hardware failure or some kind of random disk corruption.
Note that Mercurial is not a backup system and does not provide redundancy. Making frequent back-ups (which in Mercurial’s case is as easy as a ‘hg push’) is the only way to make sure you don’t lose your precious code.
An alternate cause that I feel I should warn you about are virus scanners or the Windows indexing service. These lock files in a certain way that prevents them from being deleted during short time windows. Although Mercurial does its best to be robust, it is hard to defend against all cases. It is recommended to white-list your repositories, see this note.
I found a solution (Thanks to Laurens Holst) ONLY if you have a clean bakcup (with no error) including the issue revision.
In my problem rev issue is 7 and i have a backup until rev 18.
Steps :
Clone the backup repository at the last common rev (here it is 18)
Pull broken repository revs into cloned one (you have now two heads but no modifications on the working directory of course)
Update cloned repository to the most recent revision (tip)
You have now a working .hg dir :)
I just did hg pull on a repository and brought in some changesets. It said to run hg update, so I did. Unfortunately, when I did that, it failed with the following error message:
abort: integrity check failed on 00manifest.i:173!
When I run hg verify, it tells me there are a number of issues with things not in the manifest (with some slight path obscuring):
>hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
somewhere1/file1.aspx#172: in changeset but not in manifest
somewhere2/file1.pdf#170: in changeset but not in manifest checking files
file3.csproj#172: ee005cae8058 not in manifests
somewhere2/file1.pdf#171: 00371c8b9d95 not in manifests
somewhere3/file1.ascx#170: 5c921d9bf620 not in manifests
somewhere4/file1.ascx#172: 23acbd0efd3a not in manifests
somewhere5/file1.aspx#170: ce48ed795067 not in manifests
somewhere5/file2.aspx#171: 15d13df4206f not in manifests
1328 files, 174 changesets, 3182 total revisions
8 integrity errors encountered!
(first damaged changeset appears to be 170)
The source repository passes hg verify just fine.
Is there any way to recover from an integrity check failure or do I need to re-clone the repository completely from the source (not a huge issue in this case)? What could I have done to cause this, so I don't do it again?
Well, since the first damaged changeset is 170, you could clone your local repository to 169 and then pull from the source. That means only pulling 5 changesets.
hg clone -r 169 damagedrepo fixedrepo
cd fixedreop
hg verify
And then:
hg pull originalsource
As for manual recovery of repository corruption, this page expounds on that better than I can. See section 4:
I have found corruption once in a while before, and although the above
documentation says it is usually from user error, my instances were on
removable USB drives with empty working directories. Sometimes things
just don't get written correctly or are interfered with somehow: it's
not always user error. But I always have multiple copies I can reclone
from so I've been able to get away with basic fixing.
If the simple fix of a partial local clone and pulling from the server doesn't fix it, you're down to 2 options after backing up your changes (if any) to a bundle or patches:
Manually hacking at Mercurial's files.
Doing a new full clone from the server. Usually the easier and faster of the two.
Beware: This method will change all hashes.
Actually there is another way to recover the repository when it is corrupted like this -
You can do a complete rebuild of the repository by using the convert extension. See Section 4.5 on https://www.mercurial-scm.org/wiki/RepositoryCorruption#Recovery_using_convert_extension
First enable the convert extension by adding the following to your ~/.hgrc file
[extensions]
convert=
Then convert the bad repo to create a fixed repo:
$ hg convert --config convert.hg.ignoreerrors=True REPO REPOFIX
This worked for me when I had the experience of suddenly finding that there were missing files in the manifests - "error 255".
Try remove your file 00manifest.i from repo and next use hg remove 00manifest.i and hg commit commands. Worked for me.
What we ended up doing was making a new copy of our 'central' repository, deleting the .hg folder in this copy, creating a new repository there (hg init), and then working with this as the central repository.
Be aware however this is only an appropriate solution if you don't need your changeset history other than as a reference (which we don't). You can still use your old central repository for this purpose.
I've been using Mercurial v 1.1 for several months to version documents and other files. Yesterday it suddenly failed with the message:
waiting for lock on working directory
This happens in all projects I have under .hg control. Mercurial also thinks that all files in all projects have changed.
There is no .hg/store/lock file in the project it says it is waiting on the lock for.
The only thing that could have caused this is that Windows installed security patch on my computer overnight.
Has anyone else seen this with Mercurial?
I've had success by deleting that file .hg/wlock entirely if it exists, then everything is back to normal. If you are worried about losing something, just make a copy
For working directory, the lock is .hg/wlock. Does the file exists?
For rebuilding the dirstate (beware it won't restore changes like adds/remove/renames/copies), you can use hg debugrebuildstate.
I upgraded to hg version 1.3.1 and everything works now.
I must have had corruption in the 1.1.1 binaries (from Cygwin).
Cygwin is still on 1.1.
To find out which file is locking the directory, in your working directory:
hg debuglocks
This should give a result indicating which file is locking the directory e.g.
lock: free
wlock: (461232s)
To unlock use force:
hg debuglocks --force-wlock
or:
hg debuglocks --force-lock
for more information:
hg debuglocks -h
Note this paragraph:
Locks protect the integrity of Mercurial's data, so should be treated
with care. System crashes or other interruptions may cause locks to
not be properly released, though Mercurial will usually detect and
remove such stale locks automatically.