I just did hg pull on a repository and brought in some changesets. It said to run hg update, so I did. Unfortunately, when I did that, it failed with the following error message:
abort: integrity check failed on 00manifest.i:173!
When I run hg verify, it tells me there are a number of issues with things not in the manifest (with some slight path obscuring):
>hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
somewhere1/file1.aspx#172: in changeset but not in manifest
somewhere2/file1.pdf#170: in changeset but not in manifest checking files
file3.csproj#172: ee005cae8058 not in manifests
somewhere2/file1.pdf#171: 00371c8b9d95 not in manifests
somewhere3/file1.ascx#170: 5c921d9bf620 not in manifests
somewhere4/file1.ascx#172: 23acbd0efd3a not in manifests
somewhere5/file1.aspx#170: ce48ed795067 not in manifests
somewhere5/file2.aspx#171: 15d13df4206f not in manifests
1328 files, 174 changesets, 3182 total revisions
8 integrity errors encountered!
(first damaged changeset appears to be 170)
The source repository passes hg verify just fine.
Is there any way to recover from an integrity check failure or do I need to re-clone the repository completely from the source (not a huge issue in this case)? What could I have done to cause this, so I don't do it again?
Well, since the first damaged changeset is 170, you could clone your local repository to 169 and then pull from the source. That means only pulling 5 changesets.
hg clone -r 169 damagedrepo fixedrepo
cd fixedreop
hg verify
And then:
hg pull originalsource
As for manual recovery of repository corruption, this page expounds on that better than I can. See section 4:
I have found corruption once in a while before, and although the above
documentation says it is usually from user error, my instances were on
removable USB drives with empty working directories. Sometimes things
just don't get written correctly or are interfered with somehow: it's
not always user error. But I always have multiple copies I can reclone
from so I've been able to get away with basic fixing.
If the simple fix of a partial local clone and pulling from the server doesn't fix it, you're down to 2 options after backing up your changes (if any) to a bundle or patches:
Manually hacking at Mercurial's files.
Doing a new full clone from the server. Usually the easier and faster of the two.
Beware: This method will change all hashes.
Actually there is another way to recover the repository when it is corrupted like this -
You can do a complete rebuild of the repository by using the convert extension. See Section 4.5 on https://www.mercurial-scm.org/wiki/RepositoryCorruption#Recovery_using_convert_extension
First enable the convert extension by adding the following to your ~/.hgrc file
[extensions]
convert=
Then convert the bad repo to create a fixed repo:
$ hg convert --config convert.hg.ignoreerrors=True REPO REPOFIX
This worked for me when I had the experience of suddenly finding that there were missing files in the manifests - "error 255".
Try remove your file 00manifest.i from repo and next use hg remove 00manifest.i and hg commit commands. Worked for me.
What we ended up doing was making a new copy of our 'central' repository, deleting the .hg folder in this copy, creating a new repository there (hg init), and then working with this as the central repository.
Be aware however this is only an appropriate solution if you don't need your changeset history other than as a reference (which we don't). You can still use your old central repository for this purpose.
Related
It seems like ShelveExtension only shelves your modified files leaving untracked or deleted.
I am new to Mercurial and coming from git so for me this is not expected.
Even bigger problem I am not able to hg unshelve with what I assume is an error message.
See below:
unshelving change 'main'
temporarily committing pending changes (restore with 'hg unshelve --abort')
rebasing shelved changes
abort: uncommitted changes
Is that an expected behavior and I am just missing something?
How could I unshelve my modified files without restoring/committing/etc.?
Is there an extension which behaves exactly like git stash?
Steps to reproduce:
Environment:
OS: Windows 8
Mercurial: Mercurial Distributed SCM (version 3.0.1).
Installed as cygwin /usr/bin/hg (Tortoisehg is not installed, Windows hg is installed but not used)
Extension: ShelveExtension.
Is that an expected behavior and I am just missing something?
Yes, this is normal behavior. You need to do hg addremove (or manually hg add and hg rm the individual files) if you want Mercurial to track file creation and deletion. Renaming should be done with hg mv. This is vaguely similar to git add, except that you do not need to do it for modified files.
When you unshelve, your working directory should be clean. At the very least, it should not have any missing files (prefixed with ! in hg st) nor any modified files (prefixed with M). You can always make a temporary commit and hg strip it later.
How could I unshelve my modified files without restoring/committing/etc.?
There's no sane way to do this in the general case. What if the shelf contains changes to a file which no longer exists? If the file deletion had been committed, you could generate a patch conflict, and that's what Mercurial does. But without a commit to conflict with, there's no obvious response to this situation.
Is there an extension which behaves exactly like git stash?
Not to my knowledge, but this is beyond the scope of StackOverflow.
TLDR: I have an HG bundle with parent X, but revision X does not exist in my repo. However, I am sure that the files of revision Y are identical to revision X. How do I apply the bundle?
Background:
I use hgsubversion to interact with an SVN repo.
There were some changes I did not want to commit. hgsubversion does not support partial pushes.
I used to workaround by manually creating temporary exports/patch files, or manually restoring .orig files (result of hg revert).
In this case I committed the changes I did not want to push, then used hg strip, then pushed, then tried to use hg unbundle .hg/strip-backup/file.
Problem: hgsubversion replaces the original changeset with a new one it imports from SVN after it's committed it. Result: the changeset ID changes. This is a problem because now hg unbundle no longer works, as it relies on the parent changeset being there (but it's been stripped).
Ironically, hgsubversion itself uses strip and thus has a backup file I can use to strip the new rev, add the stripped old revision, then apply the bundle with my revisions, export the patch, strip both, and restore the SVN revision. But this sounds... extremely painful and stupid. Is there nothing better I can do?
(hg transplant doesn't seem to like the bundle without having the parent in the repo, either)
It's effectively impossible to use a bundle without the bundle's precise parent changesets. Bundles consist of compact binary deltas that can only be applied to the precise binary source. There is no 'context' available that would allow Mercurial to guess how to apply them to other revisions the way patch does. In core Mercurial, this is never an issue because changesets are never removed, but extensions like hgsubversion and mq break the rules.
(If you can recover the stripped changesets from a backup bundle in .hg/strip-backup, you can then rebase your changes and strip again.)
Background: After hitting an issue with hgsubversion pushing only 1 revision successfully, I got lazy in my commits (partial commits are possible if you update to the latest revision you want to push), and ended up starting to commit everything. So I killed the push and for the first time it failed to keep my later revisions.
I tried recover, but that was not able to find the parent commit. What worked for me was restoring the ...-backup.hg file (there was a ...-temp.hg file in strip-backup too).
The strange thing (which is why I'm answering this) is that it only gave me a warning about the parent not being there (I have no idea why)...
warning: ignoring unknown working parent d5663567bc4b!
adding changesets
adding manifests
adding file changes
added 21 changesets with 1255 changes to 941 files
(run 'hg update' to get a working copy)
BTW, I'm running Mercurial version 2.0
This is what i get when i do hg verify :
repository uses revlog format 1
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
includes/base/class/ViewInstanceAdapter.class.php#7: broken revlog! (index data/includes/base/class/ViewInstanceAdapter.class.php.i is corrupted)
warning: orphan revlog 'data/includes/base/class/ViewInstanceAdapter.class.php.i'
158 files, 61 changesets, 270 total revisions
1 warnings encountered!
1 integrity errors encountered!
(first damaged changeset appears to be 7)
I do not use Mercurial for a long time and i don't understand what this means.
(I'm on windows using TortoiseHg, and the project is local only)
As said before (although you already confirmed this doesn’t work), you should start by trying to clone the repository; if the problems are related to the dirstate this can bypass it.
Next, every clone contains a complete repository, so every clone is effectively a back-up. Don’t you have a central server or colleague or another local copy? Try cloning that, then pulling from your corrupted repository. As the first damaged changeset is reported as being no. 7 (out of 270), this should be a pretty old one so likely easy to recover, and hopefully the damage does not prevent Mercurial from pulling changesets beyond that.
A third option you could try is to run a Mercurial-Mercurial conversion on your repository (hg convert repo repo-copy); a verbatim conversion should the keep changeset IDs intact, although it will probably run into the same problem. You could also try to specify a filemap to filter out the ViewInstanceAdapter file.
Because the damaged changeset is so old, and given that Mercurial uses an append-only writing method, the probable cause for this problem is a hardware failure or some kind of random disk corruption.
Note that Mercurial is not a backup system and does not provide redundancy. Making frequent back-ups (which in Mercurial’s case is as easy as a ‘hg push’) is the only way to make sure you don’t lose your precious code.
An alternate cause that I feel I should warn you about are virus scanners or the Windows indexing service. These lock files in a certain way that prevents them from being deleted during short time windows. Although Mercurial does its best to be robust, it is hard to defend against all cases. It is recommended to white-list your repositories, see this note.
I found a solution (Thanks to Laurens Holst) ONLY if you have a clean bakcup (with no error) including the issue revision.
In my problem rev issue is 7 and i have a backup until rev 18.
Steps :
Clone the backup repository at the last common rev (here it is 18)
Pull broken repository revs into cloned one (you have now two heads but no modifications on the working directory of course)
Update cloned repository to the most recent revision (tip)
You have now a working .hg dir :)
Is it possible to backup a filesystem with many Mercurial repositories (e.g., with rsync on the filesystem) and have the backup in an inconsistent state?
The repositories are served by ssh and serves this set of requests: {push, pull, in, out, clone}. It does not have 'hg commit' applied to it directly (which has a known race condition).
Mark Drago is correct that Mercurial writes its own files in a careful order to maintain integrity. However, this is only integrity with regard to other Mercurial clients. The locking design in Mercurial allows one Mercurial process to create a new commit by writing files in this order:
filelogs (holds compressed deltas for all revisions of a given file)
manifest (has pointers back to the filelogs associated with a given changeset)
changelog (has metadata and a pointer back to the manifest for the changeset)
while other Mercurial processes will read the files in this order
changelog
manifest
filelogs
The reader will thus not see a reference to the new filelog data since the changelog is updated last in an atomic operation (a rename, which POSIX requires to be atomic).
A backup program will not know the correct order to read the Mercurial files and so it might read a filelog before it was updated by Mercurial and then read a manifest after it was updated:
rsync reads .hg/store/data/foo.i
hg writes .hg/store/data/foo.i
hg writes .hg/store/00manifest.i
hg writes .hg/store/00changelog.i
rsync reads .hg/store/00manifest.i
rsync reads .hg/store/00changelog.i
The result is a backup with a changelog that points to a manifest that points to a filelog revision that does not exist --- a corrupt repository. Running hg verify on such a repository will detect this situation:
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
foo#1: f57bae649f6e in manifests not found
1 files, 2 changesets, 1 total revisions
1 integrity errors encountered!
(first damaged changeset appears to be 1)
This tells you that the manifest of revision 1 refers to revision f57bae649f6e of the file foo, which cannot be found. It is possible to repair this situation by making a clone that excludes the bad revision 1:
$ hg clone -r 0 . ../repo-fixed
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd ../repo-fixed
$ hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
1 files, 1 changesets, 1 total revisions
So, all in all, it is not that bad if you use a general backup program to backup your Mercurial repositories. Just be aware that you might have to repair a broken repository after you restore it from backup. The changeset you lose will most likely still be on the developer's machine and he can push it again after you repair the restored repository. The Mercurial wiki has more information on repairing repository corruption.
The completely safe way to backup a repository is of course to use hg clone, but it might not be practical to integrate this with a general backup strategy.
Why don't "backup" it with just hg clone? ;-)
The short answer is: You can copy (cp, rsync, etc.) a mercurial repository without problems.
The longer answer is: https://www.mercurial-scm.org/wiki/Presentations?action=AttachFile&do=get&target=ols-mercurial-paper.pdf (in particular section 5, sub-heading "Committing Changes").
Mercurial writes out changes in an order that makes it safe for any other process to read a mercurial repository at any time. If you copy a repository to some other location while a change is being made to the repository, you'll get some of the new data, but mercurial is smart enough to ignore partially written commits. When you use the copy you made as a mercurial repository you will either see the new commit or not, there will not be any corruption.
When remotely updating a Mercurial Repository, I am getting the following error from the hg update command that is being run on the remote server after the push. I looked around online for some help for this however was unsuccessful in finding anything useful. At this point, I am hoping for some ideas and / or insight as to what would be causing this problem.
The error is just below. It occurred when pushing two changesets. One changeset included an unrelated index.html file change. The other changeset was a merge, which included the index.html change as well as the renaming of the two image files.
levinaris#server01:/home/web/repository$ hg push
pushing to ssh://10.10.1.12//home/web/repository
searching for changes`remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 2 changesets with 1 changes to 1 files
remote: abort: Operation not permitted: /home/web/repository/html/images/image.gif
remote: warning: changegroup hook exited with status 255
Additional details:
Both images are 10385 bytes in size. (yes, this error occurs on two images I have)
The two images had their names changed in changesets that were already pushed and hg updated due to case-folding collisions when attempting to pull the repositories down to Windows PCs.
The target server has the following hook in /etc/mercurial/hgrc:
[hooks]
changegroup = hg update
As a work-around, I did the following:
Deleted image.gif.
Deleted another image file that produced the error.
Ran hg update - success!
Ran hg revert html/image/image.gif
Ran hg revert html/image/otherimage.gif
At this point, I am trying to better understand the cause of this problem, so that I can ensure a solid, easy-to-use implementation in my environment. I really appreciate your help!!
After using hg --debug update in the hook, I received this output:
levinaris#server01:/home/web/repository$ hg push
pushing to /home/web/staging/repository
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 1 changes to 1 files
resolving manifests
overwrite False partial False
ancestor 58a5edb95c9a local 58a5edb95c9a+ remote 3aafb97b148c
searching for copies back to rev 6
html/index.php: remote is newer -> g
html/images/otherimage.gif.casefolding: update permissions -> e
html/images/image.gif: update permissions -> e
abort: Operation not permitted: /home/web/staging/repository/html/images/image.gif
warning: changegroup hook exited with status 255
Additional Permission Information:
All 3 files in the 2 changesets have 775 permission with the webuser:dev user:group.
My Global hgrc file has the webuser trusted
[trusted]
users = webuser
Is it possible that the permissions that file on the server were such that it couldn't be overwitten by the person doing the push?
If, for example, two different people have done that push (and thus update) the second person isn't going to be able to overwrite the files created by the first person's push triggered update.
Maybe try changing the hook to this for a test (you don't actually have those single quotes on your hook, right?):
[hooks]
changegroup = hg --debug update
If it is a permissions issue the usual fix is to put everyone who will be pushing and updating into the same group (I call mine 'hg') and then using the sticky group bit on all the directories in the repo to make sure new files have that group.