I currently work for a company that uses mercurial, but most developers use MQ instead of the new evolve flow. Therefore, most users do not have the extension enabled.
Every time I push to the main repository, .hg/store/obsstore gets automatically pushed with the obsolete changesets. This is problematic because after that all users that do not have the extension enabled get the following message:
obsolete feature not enabled but 33 markers found!
One solution, proposed in this answer, involves deleting .obsstore locally, but that's not what I want, since I'm still working with evolve and that means I expect the obsolete changesets to remain hidden. Not only that, but obsolete markers and changesets should be available on the server repo so that we keep the benefits of using evolve such as intelligent conflict solving.
I'd expect a way to either not push the obsolete changesets to the server repo, or (the correct way) for the server to not push the obsolete data to clients without evolve enabled.
Is this possible? If not, why not?
If you really want to use evolve locally but not exchange your obsmarkers, you can do it. I will tell you how but be aware that you will have the exact same set of problems as with using stripping and rebase without evolve. If you push a previous version of a changeset and pushed the new version, the server will have the two versions at the same time.
If you want to use Evolve only locally you can add these lines in your config files:
[experimental]
createmarkers=True
allowunstable=True
exchangeopt=False
I found a couple of workarounds (hacks) to this problem, since it looks like mercurial has no way to solve this issue as of the writing of this answer:
Per-repository evolve extension
Remove the evolve extension from the ~/.hgrc file, enable it on a per-repository basis, and then use SSH to point to the repository (in our case it is on NFS, which is the source of the problem).
That is, on your repository's hgrc you'd have:
# This is <repo>/.hg/hgrc
[paths]
default = ssh://localhost//path/to/origin/repo
[extensions]
evolve =
and on ~/.hgrc:
[extensions]
# evolve = => Disabled on purpose
Removing obsolete markers using hooks
Just use a hook to remove the .obspurge file on your repository's hgrc:
# This is <repo>/.hg/hgrc
[alias]
obspurge = !echo "Purging obsolete markers" && rm /path/to/origin/repo/.hg/store/obsstore
[hooks]
post-push = hg obspurge
Related
How can I hinder mercurial from putting changesets to phase “public” on push operations? I want them to stay “draft”.
I rebase and histedit a lot, and the repository I push to is for me only. And having to change the phase all the time is a nuisance.
What the documentation does not clearly reveal is:
The phase-change on push is not a purely local decision. – After “uploading” the changesets, the client asks the server for updates regarding the phases of the commits, and the server is usually telling that they are now “public”.
Thus, the .hgrc-snippet
[phases]
publish = False
has to be put on the server, which inhibits the usual phase-change there. The server will then report the phases back the same way they were pushed.
Bitbucket has an option for this under Settings → Repository details → Phases.
The most direct way to keep the phase at draft is to configure the remote server as "non-publishing", as you have already discovered.
But there is a second way, which may be useful to some if the destination server cannot be set to "non-publishing" for any reason: Use pull instead of push. Pulling is read-only, so if you can set up your workflow (e.g. through a local alias) so that the remote pulls changes from your local repo, they'll remain in phase draft.
https://www.mercurial-scm.org/wiki/Phases
A repository is "publishing" by default. To make a repository non-publishing, add these lines to its hgrc configuration:
[phases]
publish = False
Short answer: nohow
If you want rewrite history both locally and on push-target, you have to
enable (on both sides)
understand
use Evolve Extension (still experimental)
I am developing a web database that is already in use for about a dozen separate installations, most of which I also manage. Each installation has a fair bit of local configuration and customization. Having just switched to mercurial from svn, I would like to take advantage of its distributed nature to keep track of local modifications. I have set up each installed server as its own repo (and configured apache not to serve the .hg directories).
My difficulty is that the development tree also contains local configuration, and I want to avoid placing every bit of it in an unversioned config file. So, how do I set things up to avoid propagating local configuration to the master repo and to the installed copies?
Example: I have a long config.ini file that should be versioned and distributed. The "clean" version contains placeholders for the database connection parameters, and I don't want the development server's passwords to end up in the repositories for the installed copies. But now and then I'll make changes (e.g., new defaults) that I do need to propagate. There are several files in a similar situation.
The best I could work out so far involves installing mq and turning the local modifications into a patch (two patches, actually, with logically separate changesets). Every time I want to commit a regular changeset to the local repo, I need to pop all patches, commit the modifications, and re-apply the patches. When I'm ready to push to the master repo, I must again pop the patches, push, and re-apply them. This is all convoluted and error-prone.
The only other alternative I can see is to forget about push and only propagate changesets as patches, which seems like an even worse solution. Can someone suggest a better set-up? I can't imagine that this is such an unusual configuration, but I haven't found anything about it.
Edit: After following up on the suggestions here, I'm coming to the conclusion that named branches plus rebase provide a simple and workable solution. I've added a description in the form of my own answer. Please take a look.
From your comments, it looks like you are already familiar with the best practice for dealing with this: version a configuration template, and keep the actual configuration unversioned.
But since you aren't happy with that solution, here is another one you can try:
Mercurial 2.1 introduced the concept of Phases. The phase is changeset metadata marking it as "secret", "draft" or "public". Normally this metadata is used and manipulated automatically by mercurial and its extensions without the user needing to be aware of it.
However, if you made a changeset 1234 which you never want to push to other repositories, you can enforce this by manually marking it as secret like this:
hg phase --force --secret -r 1234
If you then try to push to another repository, it will be ignored with this warning:
pushing to http://example.com/some/other/repository
searching for changes
no changes found (ignored 1 secret changesets)
This solution allows you to
version the local configuration changes
prevent those changes from being pushed accidentally
merge your local changes with other changes which you pull in
The big downside is of course that you cannot push changes which you made on top of this secret changeset (because that would push the secret changeset along). You'll have to rebase any such changes before you can push them.
If the problem with a versioned template and an unversioned local copy is that changes to the template don't make it into the local copies, how about modifying your app to use an unversioned localconfig.ini and fallback to a versioned config.ini for missing parameters. This way new default parameters can be added to config.ini and be propagated into your app.
Having followed up on the suggestions here, I came to the conclusion that named branches plus rebase provide a simple and reliable solution. I've been using the following method for some time now and it works very well. Basically, the history around the local changes is separated into named branches which can be easily rearranged with rebase.
I use a branch local for configuration information. When all my repos support Phases, I'll mark the local branch secret; but the method works without it. local depends on default, but default does not depend on local so it can be pushed independently (with hg push -r default). Here's how it works:
Suppose the main line of development is in the default branch. (You could have more branches; this is for concreteness). There is a master (stable) repo that does not contain passwords etc.:
---o--o--o (default)
In each deployed (non-development) clone, I create a branch local and commit all local state to it.
...o--o--o (default)
\
L--L (local)
Updates from upstream will always be in default. Whenever I pull updates, I merge them into local (n is a sequence of new updates):
...o--o--o--n--n (default)
\ \
L--L--N (local)
The local branch tracks the evolution of default, and I can still return to old configurations if something goes wrong.
On the development server, I start with the same set-up: a local branch with config settings as above. This will never be pushed. But at the tip of local I create a third branch, dev. This is where new development happens.
...o--o (default)
\
L--L (local)
\
d--d--d (dev)
When I am ready to publish some features to the main repository, I first rebase the entire dev branch onto the tip of default:
hg rebase --source "min(branch('dev'))" --dest default --detach
The previous tree becomes:
...o--o--d--d--d (default)
\
L--L (local)
The rebased changesets now belong to branch default. (With feature branches, add --keepbranches to the rebase command to retain the branch name). The new features no longer have any ancestors in local, and I can publish them with push -r default without dragging along the local revisions. (Never merge from local into default; only the other way around). If you forget to say -r default when pushing, no problem: Your push gets rejected since it would add a new head.
On the development server, I merge the rebased revs into local as if I'd just pulled them:
...o--o--d--d--d (default)
\ \
L--L-----N (local)
I can now create a new dev branch on top of local, and continue development.
This has the benefits that I can develop on a version-controlled, configured setup; that I don't need to mess with patches; that previous configuration stages remain in the history (if my webserver stops working after an update, I can update back to a configured version); and that I only rebase once, when I'm ready to publish changes. The rebasing and subsequent merge might lead to conflicts if a revision conflicts with local configuration changes; but if that's going to happen, it's better if they occur when merge facilities can help resolve them.
1 Mercurial have (follow-up to comments) selective (string-based) commit - see Record Extension
2 Local changes inside versioned public files can be easy received with MQ Extension (I do it for site-configs all time). Your headache with MQ
Every time I want to commit a regular changeset to the local repo, I
need to pop all patches, commit the modifications, and re-apply the
patches. When I'm ready to push to the master repo, I must again pop
the patches, push, and re-apply them.
is a result of not polished workflow and (some) misinterpretation. If you want commit without MQ-patches - don't do it by hand. Add alias for commit, which qop -all + commit and use this new command only. And when you push, you may don't worry about MQ-state - you push changesets from repo, not WC state. Local repo can also be protected without alias by pre-commit hook checking content.
3 You can try LocalBranches extension, where your local changes stored inside local branches (and merge branches on changes) - I found this way more troublesome, compared to MQ
What are all steps required to validate commit message with set of regular expressions?
We want to work in semi-centralized set-up so I need a solution for the developer clone (local repository) and for our central clone (global repository). I read about Mercurial Hooks but I am a little bit lost how to put all things together.
For local repository I need a way to distribute validation script across my developers. I know that hooks do not propagate when cloning so I need to a way to "enable" them in each fresh clone. It would be done as a part of our PrepareEnvironement.bat script that we run anyway on each clean clone.
To be double safe I need similar validation on my global repository. It should not be possible to push into global repository commit that are not validating. I can configure it manually - it is one time job.
I am on Windows so installing anything except TortoiseHG should not be required. It was already a fight to get Mercurial deployed. Any other dependencies are not welcomed.
You can use the Spellcheck example as a starting point. In each developer's configuration, you need to use the following hooks:
pretxnchangegroup - Runs after a group of changesets has been brought into local from another repository, but before it becomes permanent.
pretxncommit - Runs after a new changeset has been created in local, but before it becomes permanent.
For the centralized repo, I think you only need the pretxnchangegroup hook unless commits can happen on the server, too. However, you will need the Histedit extension for each of the developers if the remote repo is the one rejecting one or more of the changesets being pushed. This extension allows them to "edit" already committed changesets. I would think in most cases, the local hooks will catch the issue, but like you said, "just in case."
More details about handling events with hooks can be found in the Hg Book.
I am new to Mercurial HG. My friends created a repo and I am going to use it.
I installed TortoiseHG and trying to get the latest code. I found that when using Clone operation, it will pull all code to my local, including the histories (Am I right?). This is not needed for me. I just wanna get the latest code. Is there an operation for this?
In short, no.
In a bit longer: Mercurial doesn't yet support “shallow” clones where you only get part of the history. So each time you clone you pull in the entire repository with all changesets.
Additionally, unlike Subversion, there is no way to make a “narrow” clone where you only checkout a portion of a repository. For example, if a repository has directories foo/ and bar/, there is no way to get only the bar/ directory. In other words, Mercurial always operates on project-wide snapshots.
The easiest way to achieve what you want:
hg archive [destination folder]
Once you cloned a repository, to get the code of the "tip" (the last version of the current branch - the default one if not precised) you just need to update.
You have an update action in TortoiseHG. Once done, you can look at the files in the folder.
If you wanted another state of the repository (an old version, or an old tagged state) then it's still the update command, with other parametters (see the docs or the TortoiseHG interface).
If you only want the latest code, and you don't intend to do anything related to the repository with it, like commit, or diff to older versions, or whatever, then you it depends on where you got the code from and how.
If he is using one of the hosting services, like bitbucket, there's usually a download link which gives you just the source code.
For instance, if you go here, there's a "Get source" link up and to the right which gives you a few choices in the file format (zip or whatnot.)
If you got the files somewhere else, you need to explore the interface you got them from. Try just pasting the link you cloned from into your browser and see what you get.
Sure. Clone the repository, then delete the .hg subdirectory.
I might be a bit late but actually it is possible to forget some history with Mercurial. You just need to enable convert extension from Your mercurial.ini file or .hgrc file.
[extensions]
hgext.convert=
Now you are able to use convert extension to "clone" only changesets starting from the revision specified.
hg convert --config convert.hg.startrev=[wheretostart] path_to_full_history_repo path_to_new_repo
Just note that this is not the same operation with hg clone. That's why the source repository must be a local repository. For example if we have a repository in folder MyProject and we want to forget all the changes done before revision 100. We can use the following command:
hg convert --config convert.hg.startrev=[100] MyProject MyShrinkedProject
If You are going to use this shrunken repository on a "central server" remember to take care of that everybody clones it before they continue working. Repositories are not compatible with each other anymore.
Mercurial now supports shallow clone using remotefilelog extension. Extension is bundled with mercurial probably since version 4.9. Older versions need to download the extension e.g. from github.
You have to enable it on the server e.g:
[extensions]
remotefilelog =
[remotefilelog]
server = True
serverexpiration = 14
and on client
[extensions]
remotefilelog =
[remotefilelog]
cachepath = /some/path
cachelimit = 5 GB
Than you can do shallow clone with much smaller footprint a and faster clone speed:
hg clone --shallow ssh://user#server/repo
I want to do the equivalent of svn export REMOTE_URL with a mercurial repository. What I want at the end is an unversioned snapshot of the repository at the remote URL, but without cloning all of the changesets over to my local machine.
Also, I want to be able to specify a tag in the remote repository to pick this from. If it's not obvious, I'm building a release management tool that pulls from a canonical mercurial repository to build a release file, and it's slow right now because some projects have large, multiple-version binary files committed.
Is this possible? How would one go about it?
Its usually easier (if the remote HG is using the hgweb interface) to just visit the repo in your browser and download a .tgz / .zip / .bz2 of the tip revision. You'll see the links if the remote HG supports this.
If you want the repository, you need all of the revisions that went into the current tip for it to be at all functional.
There are options to hg clone that allow you to fetch a repository up to a certain revision, but none (that I could find) that allow you to get just the tip revision. What you are essentially asking for is a snapshot of the repo.
Edit: To Get A Snapshot
hg clone http[s]://url.to.repo repo.hg
cd repo.hg
hg archive ../repo-snapshot
cd ..
rm -rf repo.hg
The snapshot is now in repo-snapshot.
Yes, this does entail cloning the repo first, which is why I suggested seeing if the remote hgweb supports on the fly downloads of any particular revision. If it does, your problem is solved with something like curl or wget instead of HG.
If not, its good to let the original repo 'live' since you can update it again later via hg pull, then create another snapshot of a future release. This saves having to start over from scratch when cloning, especially for large repositories with lots of changes.
Also, Linux centric, but you get the gist. Of course, replace http[s] with the desired protocol as needed.
Is there any reason you can't maintain a mirror (updated in the background however often you want) of the remote repository on your local machine, then have the release management tool on your local machine run hg archive out of the local clone as necessary? If your concern is user-responsiveness, and not total bandwidth/storage consumed, this offsets the "slow" part to where you won't see it.
Tim Post noted that if you do have the hgweb CGI interface available, you can configure it to pull compressed archives down and unpack them (and the interface is consistent enough that you could script that via wget), but if you don't, core Mercurial doesn't have a lot of tools to help you, and the developers have expressed an opposition to trying to turn Mercurial into a general rsync-type client.
If you aren't afraid of playing with unofficial add-ons, you could have a look at the FTP Extension. That will force you to push from the server, however.