I develop the same system for multiple clients and I use Mercurial as the database and codebase version control.
Workflow is typically, develop on local machine, deploy to staging for testing/QA. Then I deploy to the multiple repositories on production.
However now, I have to deploy to production this week for one client and the rest in a couple of weeks time (Have to give notice to other clients). Let's call this version 10. After I deploy this week, I then need to move some new code to staging for testing. Let's call this version 11. How to I pull to production in a couple of weeks (version 10) without pulling the new changes (version 11)?
I hope that makes sense.
Such workflow is possible if you work with separate branches, e.g. if you have in default branch only that code which you consider stable enough to pull to clients and all your code so far only deemed ready for testing and QA is found in a separate branch, let's call it testing.
Thus, if you have a repository which looks like
hg log -G -T"{branch}-{rev}: {desc}\n"
# testing-4: Add some code which needs testing
|
| o default-3: Changes to be deployed to client
| |
| o default-2: Some stuff
|/
o default-1: Second changeset
|
o default-0: First changeset
you then can clone or pull only the changes which are found in the default branch, but not in your testing one:
hg clone --branch default
or
hg plull --branch default
If you later succeed with testing your code and want to deply that also to your client, first merge the code from your testing branch into default and then, like above, only pull the default branch to your clients. Thus if you do all your coding in the testing branch and only merge to default what you want to ship, then you will only ever pull what you explicit merged to default.
Of course you could also reverse the roles of branches: develop in default and have a production branch which you pull to your clients.
Related
We have a SOLUTION folder (Mercurial repository) in whitch we have a PROJECT folder, that is also a Mercurial repository.
So two repositories: one - the root(solution) folder and other - a subfolder of the root folder(the project) (yes strange but it is like this)...
Everything worked, but one day someone somehow included the SOLUTION branch into the PROJECT repository... So all the history from the Solution branch was included in parralel with the Project branch into the PROJECT repository....
Now is a little mess in the PROJECT repository... There is need to clean that repository...
Locally it worked by applying the hg strip rev XXS (where XXS was the revision number of the very first node from the freshly added Solution branch in the Project repository).
But it seems there is no strip equivalent on the server?!
Every time we'll pull incoming changes in the Project repository, the "Solution" branch will be re-imported....
Is there a way to manage it on the server side?
Of course the same solution would also work on the server. Thus you need login access to the server itself to execute the same local history operation on it. But for the default setup (publishing server) a push will never remove changesets which are present on a remote location; when you history edit your local repository, the changes will not all propagate: only additions to the graph will, but no deletions.
If such changes to the remote server are expected to be pushed, and this is a regular thing, you might want to look into use of phases and how to setup a non-publishing server, e.g. a server with mutable history: Phases#Publishing_Repository.
Mind that such a workflow also means that every single one of the people with push privilige has to change their default phase to 'draft' instead of 'public' - at least for that project.
kill the server repo. start a fresh one, then from local:
hg push -rev XXR
where XXR is the last rev you want to keep.
I am developing a web database that is already in use for about a dozen separate installations, most of which I also manage. Each installation has a fair bit of local configuration and customization. Having just switched to mercurial from svn, I would like to take advantage of its distributed nature to keep track of local modifications. I have set up each installed server as its own repo (and configured apache not to serve the .hg directories).
My difficulty is that the development tree also contains local configuration, and I want to avoid placing every bit of it in an unversioned config file. So, how do I set things up to avoid propagating local configuration to the master repo and to the installed copies?
Example: I have a long config.ini file that should be versioned and distributed. The "clean" version contains placeholders for the database connection parameters, and I don't want the development server's passwords to end up in the repositories for the installed copies. But now and then I'll make changes (e.g., new defaults) that I do need to propagate. There are several files in a similar situation.
The best I could work out so far involves installing mq and turning the local modifications into a patch (two patches, actually, with logically separate changesets). Every time I want to commit a regular changeset to the local repo, I need to pop all patches, commit the modifications, and re-apply the patches. When I'm ready to push to the master repo, I must again pop the patches, push, and re-apply them. This is all convoluted and error-prone.
The only other alternative I can see is to forget about push and only propagate changesets as patches, which seems like an even worse solution. Can someone suggest a better set-up? I can't imagine that this is such an unusual configuration, but I haven't found anything about it.
Edit: After following up on the suggestions here, I'm coming to the conclusion that named branches plus rebase provide a simple and workable solution. I've added a description in the form of my own answer. Please take a look.
From your comments, it looks like you are already familiar with the best practice for dealing with this: version a configuration template, and keep the actual configuration unversioned.
But since you aren't happy with that solution, here is another one you can try:
Mercurial 2.1 introduced the concept of Phases. The phase is changeset metadata marking it as "secret", "draft" or "public". Normally this metadata is used and manipulated automatically by mercurial and its extensions without the user needing to be aware of it.
However, if you made a changeset 1234 which you never want to push to other repositories, you can enforce this by manually marking it as secret like this:
hg phase --force --secret -r 1234
If you then try to push to another repository, it will be ignored with this warning:
pushing to http://example.com/some/other/repository
searching for changes
no changes found (ignored 1 secret changesets)
This solution allows you to
version the local configuration changes
prevent those changes from being pushed accidentally
merge your local changes with other changes which you pull in
The big downside is of course that you cannot push changes which you made on top of this secret changeset (because that would push the secret changeset along). You'll have to rebase any such changes before you can push them.
If the problem with a versioned template and an unversioned local copy is that changes to the template don't make it into the local copies, how about modifying your app to use an unversioned localconfig.ini and fallback to a versioned config.ini for missing parameters. This way new default parameters can be added to config.ini and be propagated into your app.
Having followed up on the suggestions here, I came to the conclusion that named branches plus rebase provide a simple and reliable solution. I've been using the following method for some time now and it works very well. Basically, the history around the local changes is separated into named branches which can be easily rearranged with rebase.
I use a branch local for configuration information. When all my repos support Phases, I'll mark the local branch secret; but the method works without it. local depends on default, but default does not depend on local so it can be pushed independently (with hg push -r default). Here's how it works:
Suppose the main line of development is in the default branch. (You could have more branches; this is for concreteness). There is a master (stable) repo that does not contain passwords etc.:
---o--o--o (default)
In each deployed (non-development) clone, I create a branch local and commit all local state to it.
...o--o--o (default)
\
L--L (local)
Updates from upstream will always be in default. Whenever I pull updates, I merge them into local (n is a sequence of new updates):
...o--o--o--n--n (default)
\ \
L--L--N (local)
The local branch tracks the evolution of default, and I can still return to old configurations if something goes wrong.
On the development server, I start with the same set-up: a local branch with config settings as above. This will never be pushed. But at the tip of local I create a third branch, dev. This is where new development happens.
...o--o (default)
\
L--L (local)
\
d--d--d (dev)
When I am ready to publish some features to the main repository, I first rebase the entire dev branch onto the tip of default:
hg rebase --source "min(branch('dev'))" --dest default --detach
The previous tree becomes:
...o--o--d--d--d (default)
\
L--L (local)
The rebased changesets now belong to branch default. (With feature branches, add --keepbranches to the rebase command to retain the branch name). The new features no longer have any ancestors in local, and I can publish them with push -r default without dragging along the local revisions. (Never merge from local into default; only the other way around). If you forget to say -r default when pushing, no problem: Your push gets rejected since it would add a new head.
On the development server, I merge the rebased revs into local as if I'd just pulled them:
...o--o--d--d--d (default)
\ \
L--L-----N (local)
I can now create a new dev branch on top of local, and continue development.
This has the benefits that I can develop on a version-controlled, configured setup; that I don't need to mess with patches; that previous configuration stages remain in the history (if my webserver stops working after an update, I can update back to a configured version); and that I only rebase once, when I'm ready to publish changes. The rebasing and subsequent merge might lead to conflicts if a revision conflicts with local configuration changes; but if that's going to happen, it's better if they occur when merge facilities can help resolve them.
1 Mercurial have (follow-up to comments) selective (string-based) commit - see Record Extension
2 Local changes inside versioned public files can be easy received with MQ Extension (I do it for site-configs all time). Your headache with MQ
Every time I want to commit a regular changeset to the local repo, I
need to pop all patches, commit the modifications, and re-apply the
patches. When I'm ready to push to the master repo, I must again pop
the patches, push, and re-apply them.
is a result of not polished workflow and (some) misinterpretation. If you want commit without MQ-patches - don't do it by hand. Add alias for commit, which qop -all + commit and use this new command only. And when you push, you may don't worry about MQ-state - you push changesets from repo, not WC state. Local repo can also be protected without alias by pre-commit hook checking content.
3 You can try LocalBranches extension, where your local changes stored inside local branches (and merge branches on changes) - I found this way more troublesome, compared to MQ
We are a small team of developers working with a Web Application which is published using a Web Server that is only accessible throught FTP.
Our workflow is the following one:
A developer is working out some requested feature locally
When its done, commits it and Pushes to a 'central' repository
Few times a day, one of developers publishes the files that have been changed to a testing WebSite, to let key users see how does features have been implemented.
Once in a week, we deploy to our production site
As our Webserver doesnt support SSH, we can't push changesets and update on the server, so we created a custom script which Transfers the changed files throught FTP.
Each time we use that script a new tag is created, so we know -using hg diff- the diference between tags (a release for us).
It was all fine until now, that we introduced branches in our workflow, to let a developer work on a radical changes in the code, and keep contributing in daily small changes which are published to production.
The problem is that hg diff doesnt support Branches (or seems that its still in development)
So, which would be the best way to do it ? some options we have been thinking about are:
Mounting FTP as a Volume localy (using MacFuse or similar) and use mercurial push/update But would be so sloooow.
Play around with Bundles and see if they can help us but seems quite complicated
Example
$ hg tag qa-001 /* init to see diferences QA Site */
$ hg tag prod-001 /* init to see diferences Production Site */
$ hg ci -m "working on a stable feature"
$ hg tag qa-002
$ hg ci -m "change on the stable feature"
$ hg tag qa-003
$ hg tag prod-002
$ hg ci -m "another change on stable"
$ hg pull ../CentralRepo /*Where there is another Branch with unstable files*/
With last operation, a new head is created , so now there are two heads (stable, and unstable branch)
$hg diff -r qa-003 -r tip
The Result of hg diff is showing up the Unstable Files without doing the merge
Many Thanks for your comments
In your example, you are creating tags, not (named) branches. Tags won't help you to create separate lines of development: they are just stand-alone identifiers assigned with particular revisions.
Creating branches
To start using branches, you probably want to review some tutorials, such as:
Chapter 8. Managing releases and branchy development (from Bryan O'Sullivan's book)
A Guide to Branching in Mercurial (Steve Losh)
Based on your description, you probably want to create prod and qa branches based on your current default branch, as well as any feature / topic branches you might want for radical changes.
Once you have these branches in place, it's very easy to compare them, merge between them, see what changes are pending from one to the other, and so on as your workflow demands.
Bundles
Play around with Bundles and see if they can help us but seems quite complicated
If you only have FTP access, then bundles probably won't help you. You could upload a bundle to the server via FTP, but you would need to be able to run hg on the server to unpack the bundle into a repository.
I'm new to Launchpad and Bazaar, and I'm trying to figure out what the best way is to submit bug fixes. I'm using some reasonably popular open-source software that's hosted on Launchpad, but it's not very stable. I've created my own branch of the project to stabilize it and apply just the bug fixes we need without adding other changes from ongoing development.
When I file bugs and then figure out how to fix them myself, I push the fix to our stable branch. How should I publish the fix back to the main project? I could create a patch file and attach it to the bug, or I could propose a merge for our stable branch.
If I fix multiple bugs, can I make a separate merge proposal for each one, or are they cumulative?
Update: It looks like the OpenERP project's official documentation now has guidelines for making merge proposals. I'll also leave my version here because it has some different details.
After playing with Bazaar and Launchpad for a few days and submitting a few patches and merge proposals, I thought I'd write a summary of what I found. Launchpad and Bazaar provide some powerful tools, particularly for community-driven projects, but I don't think new users are likely to fall into the pit of success yet. There are several ways to make the process slow and frustrating, so I hope this walk through helps some people avoid a few mistakes.
Here's the work flow I've learned for working on bug fixes and submitting merge proposals back to the team for a project that's hosted on Launchpad. I'm working on a GNU/Linux workstation, but I assume the Bazaar commands would be equivalent on other platforms. In this example, I'm working on one of the projects in the OpenObject project group called OpenObject Addons. The maintainer's user name is openerp. I'll put my workspace in the ~/workspace folder.
If you want to learn more about any of the commands here, use bzr help plus the command name.
Create a shared repository
Because I'm going to be creating a branch for each feature I want to contribute back to the project, I don't want to have a separate copy of the project's entire history for each branch. To avoid that, I create a shared repository and then create each branch within that. One thing to be careful of is that your repository format has to match the official branch's format to make some of the later steps work.
Check the repository format on the official branch:
bzr info http://bazaar.launchpad.net/~openerp/openobject-addons/5.0
Get the code
Now create a workspace folder that will hold any local branches on your machine - I just name it after the project. Then create a shared repository in it using the format you found on the official branch.
cd ~
mkdir workspace
cd workspace
mkdir openobject-addons
cd openobject-addons
bzr init-repo --format=rich-root-pack .
The next step is to check out the source code from the official branch. It's usually called trunk, but you might prefer to work with a stable release branch that is just being used for bug fixes. In this example, I'm going to work on the 5.0 release branch.
cd ~/workspace/openobject-addons
bzr checkout lp:~openerp/openobject-addons/5.0/ feature-x
That step is probably the slowest in the whole process for a large project, because you're copying all the code plus all the history for the entire project onto your hard drive. Note that I name the branch after the feature I'm going to work on.
Create a branch
At this point you can experiment with building and running the code on your local workstation. You can make changes to the code, but you don't have anywhere to commit them yet, because you're probably not allowed to commit directly into the official branch. To publish your code changes you need to create a public branch. If you're new to Launchpad, you'll need to create an account and register a public key first.
Once you've set up your account, you can publish your own branch as a copy of the official branch, and start working with it. The lp-login command tells bazaar what account name to use on the launchpad site, and the whoami command tells bazaar what name to use on each revision you commit. The e-mail address you use in whoami should match one of the e-mail addresses you configured for your Launchpad account.
cd ~/workspace/openobject-addons/feature-x
bzr lp-login donkirkby
bzr whoami "Don Kirkby <donkirkby#example.com>"
bzr branch --stacked --switch lp:~openerp/openobject-addons/5.0/ lp:~donkirkby/openobject-addons/feature-x
You switch to the new branch so that commits will be recorded in your local history and in your public branch. You might want to learn about the difference between a checkout and a branch. Making this a stacked branch means that it's very fast to create because it only contains the history that's not in the official branch. This blog post makes it sound like branches of public projects should default to stacked, but that hasn't worked for me. Notice that I named the branch after some feature I want to add. As bialix suggested, I create a separate branch for each feature that I will eventually propose merging back into the official branch.
Commit and make a merge proposal
Now that you have a branch, you can make code changes and commit them.
cd ~/workspace/openobject-addons/feature-x
bzr commit -m "Fixed bug lp:12345 by fleaking the woverbinate() function."
You can commit from anywhere within the branch structure, and it commits the entire branch by default. Run bzr help commit for details. You might also find bzr status and bzr diff useful.
Once you're happy with the changes and you've committed everything to your feature branch, you can go to the Launchpad web site and create a merge proposal. Here's a handy shortcut that you can run to launch the branch's web page:
cd ~/workspace/openobject-addons/feature-x
bzr lp-open
Once you create the merge proposal, Launchpad will generate a diff for it. It's well worth reviewing that diff. Sometimes I've selected the wrong branch as a target, and I only noticed because the diff had way more changes than I expected. There's also a bzr send command for merge proposals, but I haven't used it.
There's an e-mail interface for shepherding your proposal through the process, or you can just use the web site.
It's also useful to attach the branch to the bug so that other people can use it like a patch on their own systems.
Ongoing changes
If you work on several features and the maintainer isn't very speedy at reviewing your proposals, it's probably worth setting up your own mainline branch. This branch collects all your features together and it holds the code that you would run on your servers. It's also useful if the official branch isn't very stable and you want to stabilize a branch for your production environment. Then you can decide when to upgrade to the latest, and when to take specific patches for bugs that are hurting your users.
The first step is to create another branch that is stacked on the official branch:
cd ~/workspace/openobject-addons
bzr checkout lp:~openerp/openobject-addons/5.0/ main
cd main
bzr branch --stacked --switch lp:~openerp/openobject-addons/5.0/ lp:~donkirkby/openobject-addons/main
Now there are two sources of changes you'll need to merge from. First, merging from a feature or bug fix branch:
cd ~/workspace/openobject-addons/main
bzr merge lp:~donkirkby/openobject-addons/feature-x/
bzr commit -m "Merged feature x"
Of course, if you still have a local copy of the feature branch, it will be faster to do a local merge:
cd ~/workspace/openobject-addons/main
bzr merge ../feature-x
bzr commit -m "Merged feature x"
Second, you'll occassionally want to merge the latest and greatest from the official branch:
cd ~/workspace/openobject-addons/main
bzr merge --remember lp:~openerp/openobject-addons/5.0/
bzr commit -m "Merged from 5.0 branch"
After using --remember when you merge from the official branch, you can just use bzr merge on its own to merge from the official branch. If the project uses tags to mark release points, you can view a list of tags and merge from a tag.
cd ~/workspace/openobject-addons/main
bzr tags -d lp:~openerp/openobject-addons/5.0/
bzr merge -r tag:5.0.7rc2
Such policy (using merge proposals or patches) should be defined by core developers or maintainers of the project itself. But as general rule using separate branches for each fix is preferable way over just plain patches.
Because plain patches may become out-of-date when trunk branch development moving forward. When you keep the branch for the fix then you can update your fix according to recent changes in trunk.
Fixes in branches are simpler to analyze, because other developers can see all your intermediate steps for fixing the problem, or how your fix has evolved over the time as trunk changed.
Also patches attached to bug reports often tend to be missed or forgotten. While the list of all active merge proposals is prominently shown on "Branches" page of every project.
Merging your fix from your branch means the history (and annotations) will keep your name as path author, not the core dev who applied your patch.
Keep all fixes in one branch is not good for using it in merge proposal. But it's useful per se for testing all your fixes or using it as stable branch (e.g. for dogfooding). So I'd suggest to use separate (feature) branches for each separate fix, file for them separate merge proposals, and merge these branches into your stable branch as you doing today. This way you can get full freedom over applying additional changes to each your fix and then merge it again to your stable branch.
If you need to split your existing stable branch to several separate branches you can use the recipe from John Meinel described in his blog: http://jam-bazaar.blogspot.com/2009/10/refactoring-work-for-review-and-keep.html
It's my first time using a DVCS and also as a lone developer, the first time that I've actually used branches, so maybe I'm missing something here.
I have a remote repository from which I pulled the files and started working. Changes were pushed to the remote repository and of course this simple scenario works fine.
Now that my web application has some stable features, I'd like to start deploying it and so I cloned the remote repository to a new branches/stable directory outside of my working directory for the default branch and used:
hg branch stable
to create a new named branch. I created a bunch of deployment scripts that are needed only by the stable branch and I committed them as needed. Again this worked fine.
Now when I went back to my initial working directory to work on some new features, I found out that Mercurial insists on only ONE head being in the remote repository. In other words, I'd have to merge the two branches (default and stable), adding in the unneeded deployment scripts to my default branch in order to push to the main repository. This could get worse, if I had to make a change to a file in my stable branch in order to deploy.
How do I keep my named branches separate in Mercurial? Do I have to create two separate remote repositories to do so? In which case the named branches lose their value. Am I missing something here?
Use hg push -f to force the creation of a new remote head.
The reason push won't do it by default is that it's trying to remind you to pull and merge in case you forgot. What you don't want to happen is:
You and I check out revision 100 of named branch "X".
You commit locally and push.
I commit locally and push.
Now branch X looks like this in the remote repo:
--(100)--(101)
\
\---------(102)
Which head should a new developer grab if they're checking out the branch? Who knows.
After re reading the section on named branchy development in the Mercurial book, I've concluded that for me personally, the best practice is to have separate shared repositories, one for each branch. I was on the free account at bitbucket.org, so I was trying to force myself to use only one shared repository, which created the problem.
I've bit the bullet and got myself a paid account so that I can keep a separate shared repository for my stable releases.
You wrote:
I found out that Mercurial insists on only ONE head being in the remote repository.
Why do you think this is the case?
From the help for hg push:
By default, push will refuse to run if it detects the result would
increase the number of remote heads. This generally indicates the
the client has forgotten to pull and merge before pushing.
If you know that you are intentionally creating a new head in the remote repository, and this is desirable, use the -f flag.
I've come from git expecting the same thing. Just pushing the top looks like it might be one approach.
hg push -r tip