I am developing a web database that is already in use for about a dozen separate installations, most of which I also manage. Each installation has a fair bit of local configuration and customization. Having just switched to mercurial from svn, I would like to take advantage of its distributed nature to keep track of local modifications. I have set up each installed server as its own repo (and configured apache not to serve the .hg directories).
My difficulty is that the development tree also contains local configuration, and I want to avoid placing every bit of it in an unversioned config file. So, how do I set things up to avoid propagating local configuration to the master repo and to the installed copies?
Example: I have a long config.ini file that should be versioned and distributed. The "clean" version contains placeholders for the database connection parameters, and I don't want the development server's passwords to end up in the repositories for the installed copies. But now and then I'll make changes (e.g., new defaults) that I do need to propagate. There are several files in a similar situation.
The best I could work out so far involves installing mq and turning the local modifications into a patch (two patches, actually, with logically separate changesets). Every time I want to commit a regular changeset to the local repo, I need to pop all patches, commit the modifications, and re-apply the patches. When I'm ready to push to the master repo, I must again pop the patches, push, and re-apply them. This is all convoluted and error-prone.
The only other alternative I can see is to forget about push and only propagate changesets as patches, which seems like an even worse solution. Can someone suggest a better set-up? I can't imagine that this is such an unusual configuration, but I haven't found anything about it.
Edit: After following up on the suggestions here, I'm coming to the conclusion that named branches plus rebase provide a simple and workable solution. I've added a description in the form of my own answer. Please take a look.
From your comments, it looks like you are already familiar with the best practice for dealing with this: version a configuration template, and keep the actual configuration unversioned.
But since you aren't happy with that solution, here is another one you can try:
Mercurial 2.1 introduced the concept of Phases. The phase is changeset metadata marking it as "secret", "draft" or "public". Normally this metadata is used and manipulated automatically by mercurial and its extensions without the user needing to be aware of it.
However, if you made a changeset 1234 which you never want to push to other repositories, you can enforce this by manually marking it as secret like this:
hg phase --force --secret -r 1234
If you then try to push to another repository, it will be ignored with this warning:
pushing to http://example.com/some/other/repository
searching for changes
no changes found (ignored 1 secret changesets)
This solution allows you to
version the local configuration changes
prevent those changes from being pushed accidentally
merge your local changes with other changes which you pull in
The big downside is of course that you cannot push changes which you made on top of this secret changeset (because that would push the secret changeset along). You'll have to rebase any such changes before you can push them.
If the problem with a versioned template and an unversioned local copy is that changes to the template don't make it into the local copies, how about modifying your app to use an unversioned localconfig.ini and fallback to a versioned config.ini for missing parameters. This way new default parameters can be added to config.ini and be propagated into your app.
Having followed up on the suggestions here, I came to the conclusion that named branches plus rebase provide a simple and reliable solution. I've been using the following method for some time now and it works very well. Basically, the history around the local changes is separated into named branches which can be easily rearranged with rebase.
I use a branch local for configuration information. When all my repos support Phases, I'll mark the local branch secret; but the method works without it. local depends on default, but default does not depend on local so it can be pushed independently (with hg push -r default). Here's how it works:
Suppose the main line of development is in the default branch. (You could have more branches; this is for concreteness). There is a master (stable) repo that does not contain passwords etc.:
---o--o--o (default)
In each deployed (non-development) clone, I create a branch local and commit all local state to it.
...o--o--o (default)
\
L--L (local)
Updates from upstream will always be in default. Whenever I pull updates, I merge them into local (n is a sequence of new updates):
...o--o--o--n--n (default)
\ \
L--L--N (local)
The local branch tracks the evolution of default, and I can still return to old configurations if something goes wrong.
On the development server, I start with the same set-up: a local branch with config settings as above. This will never be pushed. But at the tip of local I create a third branch, dev. This is where new development happens.
...o--o (default)
\
L--L (local)
\
d--d--d (dev)
When I am ready to publish some features to the main repository, I first rebase the entire dev branch onto the tip of default:
hg rebase --source "min(branch('dev'))" --dest default --detach
The previous tree becomes:
...o--o--d--d--d (default)
\
L--L (local)
The rebased changesets now belong to branch default. (With feature branches, add --keepbranches to the rebase command to retain the branch name). The new features no longer have any ancestors in local, and I can publish them with push -r default without dragging along the local revisions. (Never merge from local into default; only the other way around). If you forget to say -r default when pushing, no problem: Your push gets rejected since it would add a new head.
On the development server, I merge the rebased revs into local as if I'd just pulled them:
...o--o--d--d--d (default)
\ \
L--L-----N (local)
I can now create a new dev branch on top of local, and continue development.
This has the benefits that I can develop on a version-controlled, configured setup; that I don't need to mess with patches; that previous configuration stages remain in the history (if my webserver stops working after an update, I can update back to a configured version); and that I only rebase once, when I'm ready to publish changes. The rebasing and subsequent merge might lead to conflicts if a revision conflicts with local configuration changes; but if that's going to happen, it's better if they occur when merge facilities can help resolve them.
1 Mercurial have (follow-up to comments) selective (string-based) commit - see Record Extension
2 Local changes inside versioned public files can be easy received with MQ Extension (I do it for site-configs all time). Your headache with MQ
Every time I want to commit a regular changeset to the local repo, I
need to pop all patches, commit the modifications, and re-apply the
patches. When I'm ready to push to the master repo, I must again pop
the patches, push, and re-apply them.
is a result of not polished workflow and (some) misinterpretation. If you want commit without MQ-patches - don't do it by hand. Add alias for commit, which qop -all + commit and use this new command only. And when you push, you may don't worry about MQ-state - you push changesets from repo, not WC state. Local repo can also be protected without alias by pre-commit hook checking content.
3 You can try LocalBranches extension, where your local changes stored inside local branches (and merge branches on changes) - I found this way more troublesome, compared to MQ
Related
Quite often in Mercurial, I happen to need local changes to a local repository, which should never ever enter the main repository. This could be (not complete list)
config files which need to differ on my PC
I want to mark my compilations so that I can distinguish versions compiled by me from official compilations
hgsub needs to differ on a PC without network access
Using TortoiseHG, my work scheme looks like this at the moment:
Commit all relevant changes, leaving the changes to kept local uncommited
Shelve changes to be kept local
Push or Pull
Deshelve
This works until I forget to exclude the changes to kept local,
which will happen sooner or later...
Then I have to waste time to restore the state before.
Is there any way to do this, e.g. by certain extensions?
Thanks for you help
You can use the secret hg phase for this.
commit your local changes
run hg phase -sf .
This will mark the current change secret. Secret changesets will not push.
A caveat with secret phases:
All secret changesets must always be on top of the stack. When you pull changes, you will need to rebase secret changes back to the top. This is typically as easy as hg rebase.
We have a SOLUTION folder (Mercurial repository) in whitch we have a PROJECT folder, that is also a Mercurial repository.
So two repositories: one - the root(solution) folder and other - a subfolder of the root folder(the project) (yes strange but it is like this)...
Everything worked, but one day someone somehow included the SOLUTION branch into the PROJECT repository... So all the history from the Solution branch was included in parralel with the Project branch into the PROJECT repository....
Now is a little mess in the PROJECT repository... There is need to clean that repository...
Locally it worked by applying the hg strip rev XXS (where XXS was the revision number of the very first node from the freshly added Solution branch in the Project repository).
But it seems there is no strip equivalent on the server?!
Every time we'll pull incoming changes in the Project repository, the "Solution" branch will be re-imported....
Is there a way to manage it on the server side?
Of course the same solution would also work on the server. Thus you need login access to the server itself to execute the same local history operation on it. But for the default setup (publishing server) a push will never remove changesets which are present on a remote location; when you history edit your local repository, the changes will not all propagate: only additions to the graph will, but no deletions.
If such changes to the remote server are expected to be pushed, and this is a regular thing, you might want to look into use of phases and how to setup a non-publishing server, e.g. a server with mutable history: Phases#Publishing_Repository.
Mind that such a workflow also means that every single one of the people with push privilige has to change their default phase to 'draft' instead of 'public' - at least for that project.
kill the server repo. start a fresh one, then from local:
hg push -rev XXR
where XXR is the last rev you want to keep.
We are using Mercurial for .Net Mvc applications.
When we merge our branch we are faced with the problem that in web.config we have a different connection string
e.g.
connectionString="data source=Dev1\MSSQLSERVER2012;Integrated Security=SSPI;AttachDBFilename=|DataDirectory|aspnetdb.mdf; User ID=sa; Password=pass1
Second web.config
connectionString="data source=Dev2\MSSQLSERVER2012;Integrated Security=SSPI;AttachDBFilename=|DataDirectory|aspnetdb.mdf; User ID=sa; Password=pass2
etc.
How we can merge our configs without editing and rewriting the config again and again.
Just ignoring the web.config file doesn't help.
Does Mercurial have an "ignore file section" option?
I have the same situation: Local configuration that I want to prevent from propagating, but I want to be able to push and pull the same files. Instead of messing with patches, I've been doing it as follows:
Let's say the regular takes place in branch default (there could be multiple branches; this is just for concreteness).
Update to a clean latest version of default, without config state.
Create a new branch, local. In this branch, commit all the local configuration as one or more changesets.
At the tip of local, create a new branch, dev. Develop your new feature here.
Your history now looks like this:
...o--o (default)
\
L--L (local)
\
d--d--d (dev)
When you're ready to push, rebase the entire dev branch onto the tip of default:
hg rebase --source "min(branch('dev'))" --dest default --detach
If your feature has its own branch name that you want to keep, add --keepbranches to the rebase options. The previous tree becomes:
...o--o--d--d--d (default)
\
L--L (local)
You can now publish the new features with push -r default without dragging along the local revisions. (Never merge from local into default; only the other way around). If you forget to say -r default when pushing, no problem: Your push gets rejected since it would add a new head.
Still on the development server, merge the rebased revs into local:
...o--o--d--d--d (default)
\ \
L--L-----N (local)
You can now create a new dev branch on top of local, and continue development.
This way my local configuration is under source contol; if it conflicts with a new feature, I can resolve it like any other merge. And if something should go wrong, I can update back to configured older versions.
My case also involves configured clones under revision management. You can see the full solution in this answer.
Not a Mercurial answer but can't you change the connection strings to refer to localhost instead of Dev1 and Dev2? You'd also need to change to Windows authentication so that you don't need to have a password in the config file.
That's how we do it in our team and we never have any problems.
Use MQ and create (one or two) patches, which convert identical, common for both branches vanilla web.config into branch-specific state
I use named branches in Mercurial.
In doing so I have created one branch called playground where I can try out various wacky experiments. I never intend to merge this branch into any others and I never want to push it to our main repository.
Since creating it, every time I do a push I am told I have added a new branch and I have to use the --new-branch flag. At this point hg push -b default (or whatever branch I'm pushing) works fine but it's annoying. Is there any way to suppress that message by letting Hg know that I am not interested in pushing that branch ever?
Starting with Mercurial 2.1 (released in February 2012), you can mark your changesets secret to keep them from being pushed to another repository. You use the new hg phase command to do this:
$ hg phase --force --secret .
This mark the current working directory parent revision (.) as being in the secret phase. Secret changesets are local to your repository: they wont be pushed or pulled. Pushing now looks like this:
$ hg push
pushing to /home/mg/tmp/repo
searching for changes
no changes to push but 2 secret changesets
There is no equivalent mechanism in older versions of Mercurial. There your best bet is to create a local clone for the changesets you don't want to push.
Update:
Mercurial 2.1 introduced the hg phase command which allows users to control what change sets are exchanged with remote repositories. #MartinGeisler answer to this question details this method.
Original Answer:
If you want to create a local branch of your code you have a couple options. You can hg clone the repository which will locally create a branch of the entire repository in your filesystem. The other alternative is you can try to use a Mercurial extension like LocalbranchExtension.
There are many ways to branch in Mercurial without using a named branch. Just find a method that suits your needs.
Further reading: http://stevelosh.com/blog/2009/08/a-guide-to-branching-in-mercurial/
In addition to the excellent answer above concerning phases, you can also specify 'default-path' (in the [paths] section of your .hgrc) to refer to the local repository:
[paths]
default = ...
default-push = .
This will cause all outgoing changesets to be compared to the specified repository. In this case, comparing outgoing changesets in your local repository TO your local repository results in nothing to push.
You can still pull/update/merge from the main repository, but no push will ever send anything back to that main repository.
If you work on multiple machines/repositories, you can set one up as described above, and configure the others to specify the 'default' path to point to the server that pushes to itself. In this way, the other machines can push/pull to your local central repository, and these changesets will never escape your carefully configured collection of repositories.
It's my first time using a DVCS and also as a lone developer, the first time that I've actually used branches, so maybe I'm missing something here.
I have a remote repository from which I pulled the files and started working. Changes were pushed to the remote repository and of course this simple scenario works fine.
Now that my web application has some stable features, I'd like to start deploying it and so I cloned the remote repository to a new branches/stable directory outside of my working directory for the default branch and used:
hg branch stable
to create a new named branch. I created a bunch of deployment scripts that are needed only by the stable branch and I committed them as needed. Again this worked fine.
Now when I went back to my initial working directory to work on some new features, I found out that Mercurial insists on only ONE head being in the remote repository. In other words, I'd have to merge the two branches (default and stable), adding in the unneeded deployment scripts to my default branch in order to push to the main repository. This could get worse, if I had to make a change to a file in my stable branch in order to deploy.
How do I keep my named branches separate in Mercurial? Do I have to create two separate remote repositories to do so? In which case the named branches lose their value. Am I missing something here?
Use hg push -f to force the creation of a new remote head.
The reason push won't do it by default is that it's trying to remind you to pull and merge in case you forgot. What you don't want to happen is:
You and I check out revision 100 of named branch "X".
You commit locally and push.
I commit locally and push.
Now branch X looks like this in the remote repo:
--(100)--(101)
\
\---------(102)
Which head should a new developer grab if they're checking out the branch? Who knows.
After re reading the section on named branchy development in the Mercurial book, I've concluded that for me personally, the best practice is to have separate shared repositories, one for each branch. I was on the free account at bitbucket.org, so I was trying to force myself to use only one shared repository, which created the problem.
I've bit the bullet and got myself a paid account so that I can keep a separate shared repository for my stable releases.
You wrote:
I found out that Mercurial insists on only ONE head being in the remote repository.
Why do you think this is the case?
From the help for hg push:
By default, push will refuse to run if it detects the result would
increase the number of remote heads. This generally indicates the
the client has forgotten to pull and merge before pushing.
If you know that you are intentionally creating a new head in the remote repository, and this is desirable, use the -f flag.
I've come from git expecting the same thing. Just pushing the top looks like it might be one approach.
hg push -r tip