Quite often in Mercurial, I happen to need local changes to a local repository, which should never ever enter the main repository. This could be (not complete list)
config files which need to differ on my PC
I want to mark my compilations so that I can distinguish versions compiled by me from official compilations
hgsub needs to differ on a PC without network access
Using TortoiseHG, my work scheme looks like this at the moment:
Commit all relevant changes, leaving the changes to kept local uncommited
Shelve changes to be kept local
Push or Pull
Deshelve
This works until I forget to exclude the changes to kept local,
which will happen sooner or later...
Then I have to waste time to restore the state before.
Is there any way to do this, e.g. by certain extensions?
Thanks for you help
You can use the secret hg phase for this.
commit your local changes
run hg phase -sf .
This will mark the current change secret. Secret changesets will not push.
A caveat with secret phases:
All secret changesets must always be on top of the stack. When you pull changes, you will need to rebase secret changes back to the top. This is typically as easy as hg rebase.
When I type hg outgoing, I get a response like this:
comparing with ssh://server:1234/path/to/repo
and a delay while it communicates over the network.
Why is this network traffic necessary? Is there something fundamental about Mercurial which means it can't remember which commits have been pushed, and which haven't?
Is there an alternative command which can give me similar information without having to communicate over the network?
As Mercurial is a distributed system, there are multiple ways for your changes to get from your local repo to the remote repo.
For example:
it is possible for someone to pull changes from you and then push those changes to the remote repo
you could actually just copy your local repo using whatever operating system you have and Mercurial would be totally unaware of that. You could then push the changes in this copy to the remote repo.
However, if you have Mercurial 2.1 or later you can use hg phase to determine which changesets have been pushed. Assuming you don't use hg phase to change the status of any changesets then the changesets with a phase of draft or secret have not been pushed and those with a phase of public have. Use
$ hg log -r "not public()"
to see unpublished changesets.
It won't catch the two examples I gave above but it will probably be good enough if you just want to know which changesets you have not pushed.
Look here or check hg help phases for instructions on how to work with phases.
A mercurial repository can potentially connect with multiple other repositories. So, I guess it needs to make sure the changes that weren't pushed from your repository, didn't arrive from another repository already.
I am developing a web database that is already in use for about a dozen separate installations, most of which I also manage. Each installation has a fair bit of local configuration and customization. Having just switched to mercurial from svn, I would like to take advantage of its distributed nature to keep track of local modifications. I have set up each installed server as its own repo (and configured apache not to serve the .hg directories).
My difficulty is that the development tree also contains local configuration, and I want to avoid placing every bit of it in an unversioned config file. So, how do I set things up to avoid propagating local configuration to the master repo and to the installed copies?
Example: I have a long config.ini file that should be versioned and distributed. The "clean" version contains placeholders for the database connection parameters, and I don't want the development server's passwords to end up in the repositories for the installed copies. But now and then I'll make changes (e.g., new defaults) that I do need to propagate. There are several files in a similar situation.
The best I could work out so far involves installing mq and turning the local modifications into a patch (two patches, actually, with logically separate changesets). Every time I want to commit a regular changeset to the local repo, I need to pop all patches, commit the modifications, and re-apply the patches. When I'm ready to push to the master repo, I must again pop the patches, push, and re-apply them. This is all convoluted and error-prone.
The only other alternative I can see is to forget about push and only propagate changesets as patches, which seems like an even worse solution. Can someone suggest a better set-up? I can't imagine that this is such an unusual configuration, but I haven't found anything about it.
Edit: After following up on the suggestions here, I'm coming to the conclusion that named branches plus rebase provide a simple and workable solution. I've added a description in the form of my own answer. Please take a look.
From your comments, it looks like you are already familiar with the best practice for dealing with this: version a configuration template, and keep the actual configuration unversioned.
But since you aren't happy with that solution, here is another one you can try:
Mercurial 2.1 introduced the concept of Phases. The phase is changeset metadata marking it as "secret", "draft" or "public". Normally this metadata is used and manipulated automatically by mercurial and its extensions without the user needing to be aware of it.
However, if you made a changeset 1234 which you never want to push to other repositories, you can enforce this by manually marking it as secret like this:
hg phase --force --secret -r 1234
If you then try to push to another repository, it will be ignored with this warning:
pushing to http://example.com/some/other/repository
searching for changes
no changes found (ignored 1 secret changesets)
This solution allows you to
version the local configuration changes
prevent those changes from being pushed accidentally
merge your local changes with other changes which you pull in
The big downside is of course that you cannot push changes which you made on top of this secret changeset (because that would push the secret changeset along). You'll have to rebase any such changes before you can push them.
If the problem with a versioned template and an unversioned local copy is that changes to the template don't make it into the local copies, how about modifying your app to use an unversioned localconfig.ini and fallback to a versioned config.ini for missing parameters. This way new default parameters can be added to config.ini and be propagated into your app.
Having followed up on the suggestions here, I came to the conclusion that named branches plus rebase provide a simple and reliable solution. I've been using the following method for some time now and it works very well. Basically, the history around the local changes is separated into named branches which can be easily rearranged with rebase.
I use a branch local for configuration information. When all my repos support Phases, I'll mark the local branch secret; but the method works without it. local depends on default, but default does not depend on local so it can be pushed independently (with hg push -r default). Here's how it works:
Suppose the main line of development is in the default branch. (You could have more branches; this is for concreteness). There is a master (stable) repo that does not contain passwords etc.:
---o--o--o (default)
In each deployed (non-development) clone, I create a branch local and commit all local state to it.
...o--o--o (default)
\
L--L (local)
Updates from upstream will always be in default. Whenever I pull updates, I merge them into local (n is a sequence of new updates):
...o--o--o--n--n (default)
\ \
L--L--N (local)
The local branch tracks the evolution of default, and I can still return to old configurations if something goes wrong.
On the development server, I start with the same set-up: a local branch with config settings as above. This will never be pushed. But at the tip of local I create a third branch, dev. This is where new development happens.
...o--o (default)
\
L--L (local)
\
d--d--d (dev)
When I am ready to publish some features to the main repository, I first rebase the entire dev branch onto the tip of default:
hg rebase --source "min(branch('dev'))" --dest default --detach
The previous tree becomes:
...o--o--d--d--d (default)
\
L--L (local)
The rebased changesets now belong to branch default. (With feature branches, add --keepbranches to the rebase command to retain the branch name). The new features no longer have any ancestors in local, and I can publish them with push -r default without dragging along the local revisions. (Never merge from local into default; only the other way around). If you forget to say -r default when pushing, no problem: Your push gets rejected since it would add a new head.
On the development server, I merge the rebased revs into local as if I'd just pulled them:
...o--o--d--d--d (default)
\ \
L--L-----N (local)
I can now create a new dev branch on top of local, and continue development.
This has the benefits that I can develop on a version-controlled, configured setup; that I don't need to mess with patches; that previous configuration stages remain in the history (if my webserver stops working after an update, I can update back to a configured version); and that I only rebase once, when I'm ready to publish changes. The rebasing and subsequent merge might lead to conflicts if a revision conflicts with local configuration changes; but if that's going to happen, it's better if they occur when merge facilities can help resolve them.
1 Mercurial have (follow-up to comments) selective (string-based) commit - see Record Extension
2 Local changes inside versioned public files can be easy received with MQ Extension (I do it for site-configs all time). Your headache with MQ
Every time I want to commit a regular changeset to the local repo, I
need to pop all patches, commit the modifications, and re-apply the
patches. When I'm ready to push to the master repo, I must again pop
the patches, push, and re-apply them.
is a result of not polished workflow and (some) misinterpretation. If you want commit without MQ-patches - don't do it by hand. Add alias for commit, which qop -all + commit and use this new command only. And when you push, you may don't worry about MQ-state - you push changesets from repo, not WC state. Local repo can also be protected without alias by pre-commit hook checking content.
3 You can try LocalBranches extension, where your local changes stored inside local branches (and merge branches on changes) - I found this way more troublesome, compared to MQ
What are all steps required to validate commit message with set of regular expressions?
We want to work in semi-centralized set-up so I need a solution for the developer clone (local repository) and for our central clone (global repository). I read about Mercurial Hooks but I am a little bit lost how to put all things together.
For local repository I need a way to distribute validation script across my developers. I know that hooks do not propagate when cloning so I need to a way to "enable" them in each fresh clone. It would be done as a part of our PrepareEnvironement.bat script that we run anyway on each clean clone.
To be double safe I need similar validation on my global repository. It should not be possible to push into global repository commit that are not validating. I can configure it manually - it is one time job.
I am on Windows so installing anything except TortoiseHG should not be required. It was already a fight to get Mercurial deployed. Any other dependencies are not welcomed.
You can use the Spellcheck example as a starting point. In each developer's configuration, you need to use the following hooks:
pretxnchangegroup - Runs after a group of changesets has been brought into local from another repository, but before it becomes permanent.
pretxncommit - Runs after a new changeset has been created in local, but before it becomes permanent.
For the centralized repo, I think you only need the pretxnchangegroup hook unless commits can happen on the server, too. However, you will need the Histedit extension for each of the developers if the remote repo is the one rejecting one or more of the changesets being pushed. This extension allows them to "edit" already committed changesets. I would think in most cases, the local hooks will catch the issue, but like you said, "just in case."
More details about handling events with hooks can be found in the Hg Book.
Is there a tool out there (preferably web-based) which would automatically detect commits to a BitBucket repository, and at that time, copy all files in the repository to a web-server via FTP?
I basically want a quick and painless way (if one exists) to set up continuous integration between my BitBucket repository and my website.
No build/compilation step would be necessary, since these are only front-end (HTML/CSS/Javascript) files.
The changegroup hook is the way to do this. See Hooks for info about what to do with it.
I've used changegroup hooks on my own hg repositories, but not in BitBucket; it's possible that the BitBucket servers are restricted in what you can do, I'm not sure. I do know a wget/curl attempt to rebuild a manual upon my server upon updating its contents in a repository on SourceForge failed for me because they've locked up their servers too tightly (sending an email from the hook would work but not http access). I would expect BitBucket to be set up better; a quick search for "bitbucket changegroup hook" doesn't seem to indicate that there are any problems with it. Try it and see!