What is a good workflow for using mercurial with two long-running branches that are slightly divergent (i.e. I never intend to entirely merge them back
together)?
In my case, this is CMS software that has been customized differently for two
different web sites. I started with projectA, and once that was working cloned it to projectB and make further tweaks to both A and B to customize them. Now I want to develop some features that show up in both A and B, without merging the site-specific customizations. How?
hg push will push everything, so that won't work
Transplant appears to give me different changeset hashes, which worries me
I feel like maybe the repositories should be set up differently, but I'm not
sure how.
As Thilo comments, the common part would be best developed (and published in A and B) as a third repo declared as a SubRepo.
That way, you respect the first two repos which are independent (one evolution on A doesn't always mean an evolution on B), and you can develop the common part in subrepo C.
A solution for Mercurial might be if you can put the different areas in files that can be in .hgignore, but then they won't be versioned, so that may not be so good.
Another way is to just use 1 repo, and set a global flag, and use template A or B depending on the flag, and / or include different code source file depending on the flag. If the difference is small, then can use if-then-else inside the same file.
You can use hg push to push the changes back together, but you don't necessarily have to merge all the changesets into the trunk. Just take the ones you want.
As stated above, a subrepo is probably the best option. Another alternative would be to have a third branch with the common work, and merge from that branch to projectA and projectB (but never back to the common branch).
This alternative is more likely to have accidents (merging the wrong way) but you might find that it is easier to set up and get working quickly.
Related
My question is similar to: Mercurial branches with different codebase
But the solution given there was to do all the work on the one customer branch and merge it to default. I don't see how that is workable in my case.
I have a project that gets distributed to 4 customers. I've setup a named branch for each customer. What is an effective way of merging changes to common code, while leaving alone some customer specific data and/or requirements?
Edit:
I have customers a,b,c. Machines M,N. and parts 1,2,3,4,5.
Right now I have subrepos a,b,c,M,N,1,2,3,4,5 and repos aM1,aM2,bM1,bN1,... . I am considering having subrepo customer (branches a,b,c). machine (branches M,N). parts (1,2,3,4,5)
Are there techniques for making change propagation easy, but also keeping some differences permanent. Maybe something like this:
TipsAndTricks.
Based on your question and clarifying comment, what you want is both impossible and not really what hg is supposed to do.
Everything I am about to say, I say as someone who has made exactly the kind of mistake you are about to make and who has lived to regret it (and slowly, painfully, try to undo it).
As I said in the comments, this seems like a faulty design. If you want something to be part of a code base only for some purposes, then that's a good indicator that what you really need is one of the following:
more configurability to turn features on or off,
a custom build script to assemble various versions of the project, or
a core code base and several customer- or purpose-specific code bases for plugins and add-ons.
You probably don't want customer-specific functionality---and definitely don't want customer-specific data---in your main project's source tree. Anytime you are about to commit a change that includes either customer configurations or code that only one customer will ever use, you need to step back and ask (1) why it should be in your core project instead of a plugin and (2) what the implications would be if another customer or third party ever got access to it by mistake.
I see at least two possible ways:
Customer-specific changes are always in separate changesets, and you move common changes from customer-specific branch into mainline not with merges, but using hg graft, cherry-picking only needed changesets
Using MQ extension, you can have customer-specific changes in MQ-patches (patch per customer), applied|unapplied individually on demand and just merge branches
In the last case separate named branches may be not needed at all (you have default and different patch applied for each customer) - at least I moved to this workflow, when I had to maintain a set of clones with only slightly different configurations and merging from default to each branch become boring
Given a central mercurial respository and mutliple teams with access to this repository, every team for once in a while has to synchronize its changes to the centralized one.
Let's say team A and team B are in the state and have the intention to pull/merge/push to the centralized repository.
A pulls.
B pulls.
A merges
B merges.
B pushes. => A has to pull and merge again before pushing!
Is there a (maybe technical) way to avoid this conflict? Maybe some kind of lock? Or is this something one has to live with? :-)
By the way this "conflict" occures in a smaller scale within the teams, too. But since the team members are in the same room one can solve this by just "shouting" his intentions throughthe room to avoid the conflict.
The simplest way may to use hg pull --rebase when pulling -- this will rebase local changes on top of the changes from the main repository and avoid having too many merges (although this actually isn't a problem for Mercurial, it may be a problem for developers trying to understand the history).
In your example, if you change B merges to B rebases, then A will still have to pull and merge again, but there will be fewer unnamed branches to try to follow.
In any case, whether rebasing or merging following a pull, I would recommend doing an immediate push so that the changes are available to everybody.
But again, I would like to emphasize that this "problematic" scenario is only problematic because of the inconvenience for the developer. This is actually the workflow DVCS's like Mercurial are designed to handle. The "standard" workflow has always been to "pull and merge/rebase" before pushing -- hence Mercurial's warning about creating new heads if you try to push without merging.
Usually you can live with it, IMHO it is common sense to work this way.
You can have a workaround, but it misuses the meaning of branches.
A and b creating branches, leaving the tip/default/master untouched.
Both can commit and push their changes, but this will create branches on the
central repository.
This could cause serious conflicts if the naming is equal, there a naming convention might help.
Someone, mostly the build engineer or system architect has to merge the changes from the different branches.
If both working on different features on the same component, it might be a good way to do so.
I would recommend you to read this chapter http://hgbook.red-bean.com/read/managing-releases-and-branchy-development.html.
This is a best practice question, and I expect the answer to be "it depends". I just hope to learn more real world scenarios and workflows.
First of all, I'm talking about different changes for the same project, so no subrepo please.
Let's say you have your code base in an hg repository. You start to work on a complicated new feature A, then a complicated bug B is reported by your trusted tester (you have testers, right?).
It's trivial if (the fix for) B depends on A. You simlply ci A then ci B.
My question is what to do when they are independent (or at least it seems now).
I can think of the following ways:
Use a separate clone for B.
Use anonymous or named branches, or bookmarks, in the same repository.
Use MQ (with B patch on top of A).
Use branched MQ (I'll explain later).
Use multiple MQ (since 1.6)
1 and 2 are covered by an excellent blog by #Steve Losh linked from a slightly related question.
The one huge advantage of 1 over the other choices is that it doesn't require any rebuild when you switch from working on one thing to the other, because the files are physically separated and independent. So it's really the only choice if, for example, A and/or B touches a header file that defines a tri-state boolean and is included by thousands of C files (don't tell me you haven't seen such a legacy code base).
3 is probably the easiest (in terms of setup and overhead), and you can flip the order of A and B if B is a small and/or urgent fix. However it can get tricky if A and B touches the same file(s). It's easy to fix patch hunks that failed to apply if A and B changes are orthogonal within the same file(s), but conceptually it's still a bit risky.
4 can make you dizzy but it's the most powerful and flexible and scalable way. I default hg qinit with -c since I want to mark work-in-progress patches and push/pull them, but it does take a conceptual leap to realize that you can branch in MQ repo too. Here are the steps (mq = hg --mq):
hg qnew bugA; make changes for A; hg qref
mq branch branchA; hg qci
hg qpop; mq up -rtip^
hg qnew bugB; make changes for B; hg qref
mq branch branchB; hg qci
To work on A again: hg qpop; mq up branchA; hg qpush
It seems crazy to take so many steps, and whenever you need to switch work you must hg qci; hg qpop; mq up <branch>; hg qpush. But consider this: you have several named release branches in the same repository, and you need to work on several projects and bug fixes at the same time for all of them (you'd better get guaranteed bonus for this kind of work). You'd get lost very soon with the other approaches.
Now my fellow hg lovers, are there other/better alternatives?
(UPDATE) qqueue almost makes #4 obsolete. See Steve Losh's elegant description here.
I would always use named branches, because that lets Mercurial do its job: to keep your project history, and to remember why you made which changes in what order to your source code. Whether to have one clone or two sitting on your disk is generally an easy one, given my working style, at least:
Does your project lack a build process, so that you can test and run things right from the source code? Then I will be tempted to have just one clone, and hg up back and forth when I need to work on another branch.
But if you have a buildout, virtualenv, or other structure that gets built, and that might diverge between the two branches, then doing an hg up then waiting for the build process to re-run can be a big pain, especially if things like setting up a sample database are involved. In that case I would definitely use two clones, one sitting at the tip of trunk, and one sitting at the tip of the emergency feature branch.
It seems like there's no more or better choices than the ones I listed in the question. So here they are again.
Use one clone per project.
Pros: total separation, thus no rebuild when switching projects.
Cons: toolchain needs to switch between two clones.
Use anonymous or named branches, or bookmarks, in the same repository.
Pros: standard hg (or any DVCS) practice; clean and clear.
Cons: must commit before switching and rebuild after.
Use MQ with one patch (or multiple consecutive patches) per project.
Pros: simple and easy.
Cons: must qrefresh before switching and rebuild after; tricky and risky if projects are not orthogonal.
Use one MQ branch (or qqueue in 1.6+) per project.
Pros: ultra flexible and scalable (for the number of concurrent projects)
Cons: must qrefresh and qcommit before switching and rebuild after; feels complicated.
Like always, there's no silver bullet, so pick and choose the one right for the job.
(UPDATE) For anyone who's in love with MQ, using MQ on top of regular branches (#2 + #3) is probably the most common and preferable practice.
If you have two concurrent projects with baseline on two branches (for example next release and current release), it's trivial to hop between them like this:
hg qnew; {coding}; hg qrefresh; {repeat}
hg qfinish -a
hg update -r <branch/bookmark/rev>
hg qimport -r <rev>; {repeat}
For the last step, qimport should add a -a option to import a line of changesets at once. I hope Meister Geisler notices this :)
So the question is, at the point when you are told to stop working on feature A, and begin independent feature B, what alternative options are there, for: How to manage concurrent development with mercurial?
Let's look at the problem with concurrency removed, the same way you write threaded code- define a simple work flow for solving any problem given to you, and apply it to each problem. Mercurial will join the work, once it's done. So, programmer A will work on feature A. Programmer B will work on feature B. Both just happen to be you. (If only we had multi-core brains:)
I would always use named branches, because that lets Mercurial do its job: to keep your project history, and to remember why you made which changes in what order to your source code.
I agree with Brandon's sentiment, but I wonder if he overlooked that feature A has not been tested? In the worst case, the code compiles and passes unit tests, but some methods implement the previous requirements, and some methods implement the new ones. A diff against the previous check-in is the tool I would use to help me get back on track with feature A.
Is your code for feature A at a point when you would normally check it in? Switching from feature A to working on feature B is not a reason to commit code to the head or to a branch. Only check in code that compiles and passes your tests. My reason is, if programmer C needs to begin feature C, a fresh checkout of this branch is no longer the best place to start. Keeping your branch heads healthy, means you can respond quickly, with more reliable bug fixes.
The goal is to have your (tested and verified) code running, so you want all your code to end up merged into the head (of your development and legacy branches). My point seems to be, I've seen branching used inefficiently: code becomes stale and then not used, the merge becomes harder than the original problem.
Only your option 1 makes sense to me. In general:
You should think your code works, before someone else sees it.
Favor the head over a branch.
Branch and check-in if someone else is picking up the problem.
Branch if your automated system or testers need your code only.
Branch if you are part of a team, working on a problem. Consider it the head, see 1-4.
With the exception of config files, the build processes should be a checkout and a single build command. It should not be any more difficult to switch between clones, than for a new programmer to join the project. (I'll admit my project needs some work here.)
Subversion shop considering switching to Mercurial, trying to figure out in advance what all the complaints from developers are going to be. There's one fairly common use case here that I can't see how to handle.
I'm working on some largish feature, and I have a significant part of the code -- or possibly several significant parts of the code -- in pieces all over the garage floor, totally unsuitable for checkin, maybe not even compiling.
An urgent bugfix request comes in. The fix is nice and local and doesn't touch any of the code I've been working on.
I make the fix in my working copy.
Now what?
I've looked at "Mercurial cherry picking changes for commit" and "best practices in mercurial: branch vs. clone, and partial merges?" and all the suggestions seem to be extensions of varying complexity, from Record and Shelve to Queues.
The fact that there apparently isn't any core functionality for this makes me suspect that in some sense this working style is Doing It Wrong. What would a Mercurial-like solution to this use case look like?
Edited to add: git, by contrast, seems designed for this workflow: git add the bugfix files, don't git add anything else (or git reset HEAD anything you might have already added), git commit.
Here's how I would handle the case:
have a dev branch
have feature branches
have a personal branch
have a stable branch.
In your scenario, I would be committing frequently to my branch off the feature branch.
When the request came in, I would hg up -r XYZ where XYZ is the rev number that they are running, then branch a new feature branch off of that(or up branchname, whatever).
Perform work, then merge into the stable branch after the work is tested.
Switch back to my work and merge up from the top feature branch commit node, thus integrating the two streams of effort.
Lots of useful functionality for Mercurial is provided in the form of extensions -- don't be afraid to use them.
As for your question, record provides what you call partial commits (it allows you to select which hunks of changes you want to commit). On the other hand, shelve allows to temporarily make your working copy clean, while keeping the changes locally. Once you commit the bug fix, you can unshelve the changes and continue working.
The canonical way to go around this (i.e. using only core) would probably be to make a clone (note that local clones are cheap as hardlinks are created instead of copies).
You would clone the repository (i.e. create a bug-fix branch in SVN terms) and do the fix from there.
Alternatively if it really is a quick fix you can use the -I option on commit to explicitly check-in individual files.
Like any DVCS, branching is your friend. Branching a repository multiple ways is the bread and butter of these system. Here's a git model you might consider adopting that works quite well with Mercurial, also.
In addition to what Santa said about branching being your friend...
Small-granularity commits are your friend. Rather than making lots of code changes in a single commit, make each logically self-contained code change in its own commit. Then it will be a lot easier to cherry-pick changes to merge between branches.
Don't use Mercurial without using the Mq Extension (it comes pre-packaged in the default installation). In addition to solving your specific problem, it solves a lot of other general problems and really should be the default way that you work (especially if you're using an IDE that doesn't integrate directly with Hg, making switching branches on the fly a difficult way to work).
...so I've gotten used to the simple stuff with Mercurial (add, commit, diff) and found out about the .hgignore file (yay!) and have gotten the hang of creating and switching between branches (branch, update -C).
I have two major questions though:
If I'm in branch "Branch1" and I want to pull in some but not all of the changes from branch "Branch2", how would I do that? Particularly if all the changes are in one subdirectory. (I guess I could just clone the whole repository, then use a directory-merge tool like Beyond Compare to pick&choose my edits. Seems like there ought to be a way to just isolate the changes in one file or one directory, though.)
Switching between branches with update -C seems so easy, I'm wondering why I would bother using clone. I can only think of a few reasons (see below) -- are there some other reasons I'm missing?
a. if I need to act on two versions/branches at once (e.g. do a performance-metric diff)
b. for a backup (clone the repository to a network drive in a physically different location)
c. to do the pick&choose merge like I've mentioned above.
I use clone for:
Short-lived local branches
Cloning to different development machines and servers
The former use is pretty rare for me - mainly when I'm trying an idea I might want to totally abandon. If I want to merge, I'll want to merge ALL the changes. This sort of branching is mainly for tracking different developers' branches so they don't disturb each other. Just to clarify this last point:
I keep working on my changes and pull my fellow devs changes and they pull mine.
When it's convenient for me I'll merge ALL of the changes from one (or all) of these branches into mine.
For feature branches, or longer lived branches, I use named branches which are more comfortably shared between repositories without merging. It also "feels" better when you want to selectively merge.
Basically I look at it this way:
Named branches are for developing different branches or versions of the app
Clones are for managing different contributions to the same version of the app.
That's my take, though really it's a matter of policy.
For question 1, you need to be a little clearer about what you mean by "changes". Which of these do you mean:
"I want to pull some, but not all, of the changesets in a different branch into this one."
"I want to pull the latest version of some, but not all, of the files in a different branch into this one."
If you mean item 1, you should look into the Transplant extension, specifically the idea of cherrypicking a couple of changesets.
If you mean item 2, you would do the following:
Update to the branch you want to pull the changes into.
Use hg revert -r <branch you want to merge> --include <files to update> to change the contents of those files to the way they are on the other branch.
Use hg commit to commit those changes to the branch as a new changeset.
As for question 2, I never use repository clones for branching myself, so I don't know. I use named branches or anonymous branches (sometimes with bookmarks).
I have another option for you to look into: mercurial queues.
The idea is, to have a stack of patches (no commits, "real" patches) ontop of your current working directory. Then, you can add or remove the applied patches, add one, remove it, add another other one, etc. One single patch or a subset of them ends up to be a new "feature" as you probably want to do with branches. After that, you can apply the patch as usual (since it is a change). Branches are probably more useful if you work with somebody else... ?